Nov 04 01:52:47.046265 ip-10-0-134-176 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Nov 04 01:52:47.046279 ip-10-0-134-176 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Nov 04 01:52:47.046288 ip-10-0-134-176 systemd[1]: kubelet.service: Failed with result 'resources'. Nov 04 01:52:47.046683 ip-10-0-134-176 systemd[1]: Failed to start Kubernetes Kubelet. Nov 04 01:52:57.137269 ip-10-0-134-176 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Nov 04 01:52:57.137291 ip-10-0-134-176 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot f5eae5a24c9e41d4bc8000af8495082f -- Nov 04 01:54:54.215752 ip-10-0-134-176 systemd[1]: Starting Kubernetes Kubelet... Nov 04 01:54:54.564353 ip-10-0-134-176 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:54:54.564353 ip-10-0-134-176 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 01:54:54.564353 ip-10-0-134-176 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:54:54.564353 ip-10-0-134-176 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:54:54.569028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.568830 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.569028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569027 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.569028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569031 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.569028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569034 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.569028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569037 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569040 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569046 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569049 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569051 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569054 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569056 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569059 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569063 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569069 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569073 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569075 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569078 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569081 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569083 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569087 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569090 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569092 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569095 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.569222 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569097 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569100 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569102 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569105 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569108 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569111 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569113 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569116 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569118 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569121 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569124 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569126 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569129 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569131 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569135 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569138 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569141 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569143 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569146 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569148 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.569707 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569153 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569156 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569158 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569161 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569163 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569166 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569169 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569171 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569174 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569177 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569180 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569182 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569185 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569188 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569190 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569193 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569195 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569198 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569200 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.570170 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569203 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569205 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569208 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569210 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569213 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569215 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569218 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569221 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569224 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569227 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569230 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569233 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569235 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569239 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569243 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569246 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569249 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569252 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569255 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.570643 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569258 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569260 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569264 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569267 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569270 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569273 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569275 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569277 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569282 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569284 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569287 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569290 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569292 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569295 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569297 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569300 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569302 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569305 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569307 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569310 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.571128 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569313 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569316 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569319 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569321 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.569324 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570105 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570112 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570115 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570117 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570121 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570123 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570126 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570128 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570131 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570133 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570137 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570140 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570142 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570145 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.571626 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570147 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570150 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570152 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570155 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570157 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570161 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570163 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570167 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570169 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570172 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570176 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570179 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570181 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570184 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570187 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570190 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570192 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570195 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570197 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.572097 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570201 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570203 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570205 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570208 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570210 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570213 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570215 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570218 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570220 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570223 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570226 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570230 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570234 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570237 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570240 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570243 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570245 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570248 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570251 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.572579 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570254 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570259 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570261 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570264 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570267 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570269 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570272 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570274 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570277 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570280 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570282 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570285 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570287 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570289 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570292 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570294 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570296 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570299 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570301 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.573028 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570304 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570306 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570308 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570311 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570313 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570316 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570319 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570321 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570324 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570326 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570329 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570332 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570335 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570337 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570341 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570343 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570346 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570348 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570351 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570354 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.573495 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570357 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570362 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570366 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570370 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570375 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570378 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570382 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570387 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570391 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570395 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570400 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570403 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570406 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570409 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570411 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570977 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570984 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570988 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570992 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570994 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.573969 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570997 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.570999 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571002 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571005 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571008 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571010 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571015 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571018 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571020 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571023 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571025 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571028 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571031 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571033 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571036 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571038 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571041 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571043 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571046 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571049 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.574451 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571051 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571054 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571057 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571059 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571062 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571064 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571067 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571069 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571072 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571074 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571077 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571080 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571082 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571085 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571087 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571089 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571092 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571095 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571099 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571101 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.574941 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571104 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571106 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571109 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571111 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571114 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571116 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571119 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571122 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571124 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571126 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571129 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571132 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571135 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571138 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571140 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571143 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571146 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571148 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571150 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.575403 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571153 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571155 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571158 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571160 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571163 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571166 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571168 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571171 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571174 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571176 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571179 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571183 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571187 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571190 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571193 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571195 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571199 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571202 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571204 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.575992 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571206 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571209 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571211 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571213 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571216 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571218 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571221 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571223 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571225 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571228 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571232 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571236 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571238 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571241 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571245 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571247 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571249 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571252 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571254 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.576486 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571257 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571260 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571262 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571264 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571357 2580 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571364 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571370 2580 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571374 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571379 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571382 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571388 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571392 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571395 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571398 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571402 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571405 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571408 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571411 2580 flags.go:64] FLAG: --cgroup-root="" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571414 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571417 2580 flags.go:64] FLAG: --client-ca-file="" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571419 2580 flags.go:64] FLAG: --cloud-provider="external" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571422 2580 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571427 2580 flags.go:64] FLAG: --cluster-domain="" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571429 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 01:54:54.576935 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571433 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571436 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571439 2580 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571443 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571446 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571449 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571452 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571455 2580 flags.go:64] FLAG: --contention-profiling="false" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571458 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571461 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571464 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571481 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571485 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571489 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571492 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571495 2580 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571498 2580 flags.go:64] FLAG: --enable-server="true" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571500 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571510 2580 flags.go:64] FLAG: --event-burst="100" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571513 2580 flags.go:64] FLAG: --event-qps="50" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571516 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571519 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571522 2580 flags.go:64] FLAG: --eviction-hard="" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571526 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 01:54:54.577518 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571529 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571532 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571535 2580 flags.go:64] FLAG: --eviction-soft="" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571538 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571540 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571543 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571546 2580 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571549 2580 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571551 2580 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571554 2580 flags.go:64] FLAG: --feature-gates="" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571558 2580 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571561 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571564 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571567 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571570 2580 flags.go:64] FLAG: --healthz-port="10248" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571574 2580 flags.go:64] FLAG: --help="false" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571576 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571580 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571583 2580 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571585 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571589 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571592 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571595 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 01:54:54.578129 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571598 2580 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571601 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571603 2580 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571607 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571610 2580 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571613 2580 flags.go:64] FLAG: --kube-reserved="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571616 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571619 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571622 2580 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571625 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571628 2580 flags.go:64] FLAG: --lock-file="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571631 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571634 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571637 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571642 2580 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571645 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571648 2580 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571651 2580 flags.go:64] FLAG: --logging-format="text" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571654 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571658 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571660 2580 flags.go:64] FLAG: --manifest-url="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571663 2580 flags.go:64] FLAG: --manifest-url-header="" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571667 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571670 2580 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571674 2580 flags.go:64] FLAG: --max-pods="110" Nov 04 01:54:54.578909 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571677 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571680 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571683 2580 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571686 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571690 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571693 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571696 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571703 2580 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571706 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571709 2580 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571713 2580 flags.go:64] FLAG: --pod-cidr="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571716 2580 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571719 2580 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571722 2580 flags.go:64] FLAG: --pods-per-core="0" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571725 2580 flags.go:64] FLAG: --port="10250" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571728 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571731 2580 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0e66ea304c654fcb5" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571734 2580 flags.go:64] FLAG: --qos-reserved="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571737 2580 flags.go:64] FLAG: --read-only-port="10255" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571740 2580 flags.go:64] FLAG: --register-node="true" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571743 2580 flags.go:64] FLAG: --register-with-taints="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571747 2580 flags.go:64] FLAG: --registry-burst="10" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571750 2580 flags.go:64] FLAG: --registry-qps="5" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571752 2580 flags.go:64] FLAG: --reserved-cpus="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571755 2580 flags.go:64] FLAG: --reserved-memory="" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571759 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 01:54:54.579952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571762 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571765 2580 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571767 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571770 2580 flags.go:64] FLAG: --runonce="false" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571773 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571776 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571779 2580 flags.go:64] FLAG: --seccomp-default="false" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571781 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571784 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571788 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571792 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571795 2580 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571798 2580 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571801 2580 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571804 2580 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571806 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571811 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571814 2580 flags.go:64] FLAG: --system-cgroups="" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571817 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571822 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571825 2580 flags.go:64] FLAG: --tls-cert-file="" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571828 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571832 2580 flags.go:64] FLAG: --tls-min-version="" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571835 2580 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571838 2580 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 01:54:54.581127 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571841 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571843 2580 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571846 2580 flags.go:64] FLAG: --v="2" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571851 2580 flags.go:64] FLAG: --version="false" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571855 2580 flags.go:64] FLAG: --vmodule="" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571859 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.571862 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.571997 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572001 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572004 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572007 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572009 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572012 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572014 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572025 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572028 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572032 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572036 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572039 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572042 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.582215 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572044 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572047 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572050 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572053 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572056 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572058 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572061 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572063 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572066 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572068 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572071 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572074 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572076 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572078 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572081 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572083 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572086 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572088 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572091 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572093 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.583098 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572095 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572098 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572101 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572103 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572106 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572108 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572111 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572114 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572116 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572120 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572123 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572125 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572128 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572130 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572133 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572136 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572139 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572141 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572144 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572146 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.583650 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572148 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572151 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572153 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572156 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572158 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572161 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572163 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572166 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572168 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572171 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572173 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572176 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572179 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572181 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572183 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572187 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572190 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572193 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572196 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.584390 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572198 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572201 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572204 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572207 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572210 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572213 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572216 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572218 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572220 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572223 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572225 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572228 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572230 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572233 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572235 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572238 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572240 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572242 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572245 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.585192 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572248 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572252 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572255 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572257 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572260 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572262 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572265 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572267 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572270 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572273 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572275 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572278 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572280 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572283 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572285 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.585708 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.572313 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572349 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572353 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572356 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572359 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572362 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572365 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572368 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572372 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572375 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572378 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572380 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572383 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572385 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572388 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572390 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572393 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572395 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572398 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572401 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.586169 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572403 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572406 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572408 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572412 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572415 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572419 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572422 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572424 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572427 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572430 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572432 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572435 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572437 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572441 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572444 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572446 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572449 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572452 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572455 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.586728 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572458 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572460 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572462 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572465 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572482 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572485 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572488 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572490 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572500 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572502 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572505 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572507 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572510 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572512 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572515 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572517 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572520 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572522 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572525 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.587246 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572528 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572530 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572533 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572537 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572539 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572542 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572544 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572548 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572550 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572553 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572555 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572558 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572561 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572563 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572566 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572569 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572571 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572573 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572576 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572578 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.587790 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572581 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572584 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572586 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572589 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572591 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572594 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572596 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572599 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572601 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572604 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572607 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572609 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572611 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572614 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572616 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572620 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572622 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572625 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572627 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572633 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.588405 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572635 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572638 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572640 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572643 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572646 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572648 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572651 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572653 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.572657 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.581010 2580 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.581030 2580 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.581053 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.581061 2580 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581145 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581153 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581158 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581163 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581168 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581172 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.588921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581176 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581180 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581185 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581190 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581194 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581198 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581203 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581207 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581211 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581216 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581220 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581224 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581229 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581233 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581237 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581241 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581246 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581251 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581255 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.589708 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581259 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581264 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581268 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581273 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581277 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581282 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581286 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581291 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581295 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581302 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581308 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581313 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581317 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581322 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581326 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581330 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581334 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581339 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581343 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581348 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.590423 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581352 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581357 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581361 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581365 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581369 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581373 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581378 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581382 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581386 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581391 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581395 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581400 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581404 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581409 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581413 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581418 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581422 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581427 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581431 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581435 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.590921 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581440 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581444 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581448 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581452 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581456 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581461 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581483 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581488 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581492 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581497 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581501 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581505 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581509 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581513 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581517 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581522 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581526 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581530 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581537 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.591596 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581543 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581549 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581554 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581559 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581564 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581569 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581574 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581579 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581583 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581587 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581591 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581595 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581599 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581604 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581608 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581612 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581615 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581619 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581623 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.592338 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581627 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581631 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581636 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.581677 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581927 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581937 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581942 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581946 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581951 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581955 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581959 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581964 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581968 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581972 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581976 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581981 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.592809 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581985 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581990 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581994 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.581999 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582003 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582007 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582011 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582015 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582019 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582024 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582028 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582032 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582036 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582040 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582044 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582048 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582052 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582057 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582061 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582065 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.593231 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582070 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582076 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582083 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582087 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582091 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582096 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582100 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582104 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582108 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582112 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582117 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582121 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582125 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582129 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582135 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582140 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582146 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582151 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.593724 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582155 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582159 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582163 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582168 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582172 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582176 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582180 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582185 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582189 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582193 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582197 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582201 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582205 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582210 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582214 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582218 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582222 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582225 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582229 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.594214 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582233 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582237 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582241 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582245 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582250 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582255 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582259 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582263 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582268 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582272 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582276 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582280 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582286 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582290 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582294 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582299 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582303 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582307 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582311 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.594681 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582315 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582319 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582324 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582328 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582331 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582336 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582340 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582345 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582349 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582353 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582357 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582361 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582365 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582369 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582373 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582377 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582382 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.595391 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582386 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.582424 2580 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582499 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582513 2580 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582518 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582524 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582528 2580 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582533 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582537 2580 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582542 2580 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582547 2580 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582551 2580 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582555 2580 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582559 2580 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582563 2580 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582570 2580 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:54:54.596072 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582575 2580 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582579 2580 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582583 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582587 2580 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582592 2580 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582597 2580 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582601 2580 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582605 2580 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582610 2580 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582614 2580 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582619 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582625 2580 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582632 2580 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582637 2580 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582642 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582646 2580 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582650 2580 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582655 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582659 2580 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:54:54.596572 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582663 2580 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582668 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582672 2580 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582676 2580 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582680 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582686 2580 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582691 2580 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582697 2580 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582702 2580 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582706 2580 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582710 2580 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582715 2580 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582721 2580 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582725 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582729 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582733 2580 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582738 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582743 2580 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:54:54.597132 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582747 2580 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582751 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582755 2580 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582759 2580 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582763 2580 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582767 2580 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582771 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582775 2580 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582780 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582785 2580 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582790 2580 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582794 2580 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582798 2580 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582802 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582807 2580 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582812 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582816 2580 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582820 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582824 2580 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:54:54.597717 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582828 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582833 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582837 2580 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582842 2580 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582846 2580 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582850 2580 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582854 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582860 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582865 2580 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582870 2580 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582874 2580 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582878 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582883 2580 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582887 2580 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582891 2580 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582895 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582900 2580 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582904 2580 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582908 2580 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582912 2580 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:54:54.598256 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582916 2580 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582921 2580 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582925 2580 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582929 2580 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582933 2580 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582937 2580 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582941 2580 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582945 2580 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582951 2580 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582956 2580 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582960 2580 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582964 2580 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582968 2580 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582972 2580 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582977 2580 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:54.582981 2580 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.583056 2580 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.586074 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.587096 2580 server.go:1014] "Starting client certificate rotation" Nov 04 01:54:54.598752 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.587197 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 01:54:54.599222 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.587952 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 01:54:54.599222 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.589585 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 01:54:54.599222 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.592271 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 01:54:54.605654 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.605637 2580 log.go:25] "Validated CRI v1 runtime API" Nov 04 01:54:54.610162 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.610148 2580 log.go:25] "Validated CRI v1 image API" Nov 04 01:54:54.611007 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.610980 2580 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 01:54:54.616209 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.616190 2580 fs.go:135] Filesystem UUIDs: map[23976f5f-675a-40b9-a5f9-a29f45c91abb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e2c22dab-699c-4abd-813e-e99e726c1b55:/dev/nvme0n1p4] Nov 04 01:54:54.616276 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.616208 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0}] Nov 04 01:54:54.622569 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.622436 2580 manager.go:217] Machine: {Timestamp:2026-11-04 01:54:54.621089624 +0000 UTC m=+0.321188734 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3123821 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28e9e6a901cc9ebb20825136bf1dcb SystemUUID:ec28e9e6-a901-cc9e-bb20-825136bf1dcb BootID:f5eae5a2-4c9e-41d4-bc80-00af8495082f Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:78:27:26:be:a5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:78:27:26:be:a5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:52:c7:b4:e5:50:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 01:54:54.623194 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.623183 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 01:54:54.623274 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.623262 2580 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 01:54:54.626115 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626091 2580 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 01:54:54.626242 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626116 2580 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-176.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 01:54:54.626290 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626245 2580 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 01:54:54.626290 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626253 2580 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 01:54:54.626339 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626291 2580 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 01:54:54.626339 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.626309 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 01:54:54.627542 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.627532 2580 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 01:54:54.632091 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.632078 2580 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 01:54:54.632213 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.632203 2580 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 01:54:54.632979 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.632968 2580 kubelet.go:493] "Attempting to sync node with API server" Nov 04 01:54:54.633015 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.632983 2580 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 01:54:54.633015 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.632994 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 01:54:54.633015 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.633003 2580 kubelet.go:405] "Adding apiserver pod source" Nov 04 01:54:54.633015 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.633012 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 01:54:54.633351 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.633338 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 01:54:54.635911 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.635895 2580 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 01:54:54.642480 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.642453 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 01:54:54.643279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643268 2580 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 01:54:54.643312 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643297 2580 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 01:54:54.643456 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643446 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 01:54:54.643498 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643463 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 01:54:54.643498 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643496 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 01:54:54.643555 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643503 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 01:54:54.643555 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643508 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 01:54:54.643555 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.643514 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 01:54:54.644187 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644178 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 01:54:54.644187 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644187 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 01:54:54.644238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644195 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 01:54:54.644238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644201 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 01:54:54.644238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644214 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 01:54:54.644238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.644223 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 01:54:54.645741 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.645732 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 01:54:54.645779 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.645743 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 01:54:54.649693 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.649679 2580 server.go:1263] "Started kubelet" Nov 04 01:54:54.649787 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.649749 2580 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 01:54:54.649865 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.649777 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 01:54:54.649865 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.649840 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 01:54:54.650489 ip-10-0-134-176 systemd[1]: Started Kubernetes Kubelet. Nov 04 01:54:54.651317 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.651249 2580 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 01:54:54.652170 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.652158 2580 server.go:317] "Adding debug handlers to kubelet server" Nov 04 01:54:54.657762 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.657734 2580 kubelet.go:1679] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Nov 04 01:54:54.662215 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.662199 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 01:54:54.662846 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.662832 2580 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 01:54:54.662921 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.662902 2580 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:54:54.663649 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.663620 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 01:54:54.663724 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.663640 2580 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 01:54:54.663800 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.663772 2580 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 01:54:54.663920 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.663908 2580 reconstruct.go:97] "Volume reconstruction finished" Nov 04 01:54:54.663920 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.663920 2580 reconciler.go:29] "Reconciler: start to sync state" Nov 04 01:54:54.664636 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664618 2580 factory.go:153] Registering CRI-O factory Nov 04 01:54:54.664722 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664656 2580 factory.go:223] Registration of the crio container factory successfully Nov 04 01:54:54.664722 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664711 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 01:54:54.664722 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664720 2580 factory.go:55] Registering systemd factory Nov 04 01:54:54.664843 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664730 2580 factory.go:223] Registration of the systemd container factory successfully Nov 04 01:54:54.664843 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664745 2580 factory.go:103] Registering Raw factory Nov 04 01:54:54.664843 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.664759 2580 manager.go:1199] Started watching for new ooms in manager Nov 04 01:54:54.665859 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.665844 2580 manager.go:319] Starting recovery of all containers Nov 04 01:54:54.676878 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.676863 2580 manager.go:324] Recovery completed Nov 04 01:54:54.681403 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.681391 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.682890 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.682865 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.682963 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.682898 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.682963 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.682909 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.683432 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.683420 2580 cpu_manager.go:226] "Starting" policy="none" Nov 04 01:54:54.683432 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.683421 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.683559 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.683434 2580 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 01:54:54.683559 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.683454 2580 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 01:54:54.684585 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.684567 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.684670 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.684597 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.684670 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.684612 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.685614 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.685598 2580 policy_none.go:50] "Start" Nov 04 01:54:54.685657 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.685622 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 01:54:54.685657 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.685637 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 01:54:54.687666 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.687653 2580 policy_none.go:44] "Start" Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.732871 2580 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.743007 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.744339 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.744357 2580 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.744375 2580 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 01:54:54.748837 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.744404 2580 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 01:54:54.749055 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.748905 2580 manager.go:341] "Starting Device Plugin manager" Nov 04 01:54:54.749055 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.748936 2580 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 01:54:54.749055 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.748947 2580 server.go:87] "Starting device plugin registration server" Nov 04 01:54:54.749207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.749195 2580 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 01:54:54.749242 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.749210 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 01:54:54.749333 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.749314 2580 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 01:54:54.749403 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.749397 2580 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 01:54:54.749458 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.749405 2580 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 01:54:54.749853 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.749822 2580 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 01:54:54.749927 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.749884 2580 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:54:54.797214 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.797191 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-176.us-west-1.compute.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Nov 04 01:54:54.797214 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.797201 2580 controller.go:201] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-176.us-west-1.compute.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Nov 04 01:54:54.845291 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.845206 2580 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal"] Nov 04 01:54:54.845291 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.845282 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.846212 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.846192 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.846304 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.846225 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.846304 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.846236 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.846379 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.845297 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-176.us-west-1.compute.internal.18e4b4c178dbbc68 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-176.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-176.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:54.649654376 +0000 UTC m=+0.349753486,LastTimestamp:2026-11-04 01:54:54.649654376 +0000 UTC m=+0.349753486,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-176.us-west-1.compute.internal,}" Nov 04 01:54:54.848509 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.848493 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.848632 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.848603 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.848711 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.848639 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.849286 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849253 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.849286 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849283 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.849433 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849261 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.849433 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849320 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.849433 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849333 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.849433 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849287 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.849433 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849360 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.849907 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849893 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.849968 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849915 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.849968 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849925 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.849968 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.849943 2580 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.851426 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.851412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.851514 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.851435 2580 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:54:54.852075 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.852060 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:54:54.852148 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.852087 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:54:54.852148 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.852101 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:54:54.864959 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.864942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.865047 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.864967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.865047 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.864983 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.867032 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.867015 2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 01:54:54.867083 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.867072 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-hs8mf" Nov 04 01:54:54.884856 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.884839 2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.887594 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:54.887575 2580 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.960106 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.960082 2580 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:54.966058 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966043 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.966113 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.966113 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.966177 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966152 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.966177 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966155 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:54.966238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:54.966159 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.007912 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.007895 2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:55.055115 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.055094 2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:55.056500 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.056485 2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:55.057165 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.057149 2580 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.057201 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:55.057177 2580 kubelet_node_status.go:524] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-176.us-west-1.compute.internal\": node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:54:55.063653 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.063640 2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.123326 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.123263 2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:54:55.124124 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.124111 2580 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.184451 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.184429 2580 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:54:55.190582 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.190567 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.192720 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.192707 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:54:55.587992 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.587914 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Nov 04 01:54:55.588563 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.588046 2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 01:54:55.588563 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.588058 2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 01:54:55.588563 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.588046 2580 reflector.go:578] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:161" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:161: Unexpected watch close - watch lasted less than a second and no items received" Nov 04 01:54:55.588563 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:55.588051 2580 event.go:368] "Unable to write event (may retry after sleeping)" err="Patch \"https://ab045e258af594899a140baaafc158e1-d72efd8280e6925c.elb.us-east-2.amazonaws.com:6443/api/v1/namespaces/default/events/ip-10-0-134-176.us-west-1.compute.internal.18e4b4c17ad73816\": read tcp 10.0.134.176:60202->3.143.52.118:6443: use of closed network connection" event="&Event{ObjectMeta:{ip-10-0-134-176.us-west-1.compute.internal.18e4b4c17ad73816 default 4941 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-176.us-west-1.compute.internal,UID:,APIVersion:v1,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-176.us-west-1.compute.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-176.us-west-1.compute.internal,},FirstTimestamp:2026-11-04 01:54:54 +0000 UTC,LastTimestamp:2026-11-04 01:54:54.846240021 +0000 UTC m=+0.546339130,Count:3,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-176.us-west-1.compute.internal,}" Nov 04 01:54:55.633770 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.633748 2580 apiserver.go:52] "Watching apiserver" Nov 04 01:54:55.663030 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.663007 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Nov 04 01:54:55.873111 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.873031 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:55 +0000 UTC" deadline="2028-06-21 16:38:22.221716238 +0000 UTC" Nov 04 01:54:55.873111 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.873061 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14294h43m26.348658606s" Nov 04 01:54:55.897234 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.897214 2580 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 01:54:55.897520 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.897504 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-vdgnv","openshift-multus/multus-additional-cni-plugins-59clt","openshift-cluster-node-tuning-operator/tuned-7s8zx","openshift-multus/network-metrics-daemon-sd84m","openshift-network-diagnostics/network-check-target-nkzgd","kube-system/konnectivity-agent-75hc6","kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal","openshift-image-registry/node-ca-mbhp4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal","openshift-multus/multus-zxf47","openshift-network-operator/iptables-alerter-zd9fs","openshift-ovn-kubernetes/ovnkube-node-jm8wv","kube-system/global-pull-secret-syncer-w2257","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw"] Nov 04 01:54:55.899925 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.899910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:55.899989 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:55.899969 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:54:55.901879 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.901867 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.904320 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.904297 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:55.904414 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:55.904395 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:54:55.906793 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.906772 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:55.906876 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:55.906856 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:54:55.908979 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.908965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:55.911065 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.911051 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.913115 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.913103 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:55.915219 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.915204 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.917306 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.917294 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.919549 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.919529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:55.921892 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.921876 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.924122 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.924105 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:55.953686 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.953659 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 01:54:55.953837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.953820 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 01:54:55.953914 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.953880 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\"" Nov 04 01:54:55.960668 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.960644 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 01:54:55.960794 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.960781 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\"" Nov 04 01:54:55.960881 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.960787 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 01:54:55.962547 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.962530 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 01:54:55.962640 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.962543 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 01:54:55.962640 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.962608 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\"" Nov 04 01:54:55.962640 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.962614 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 01:54:55.964481 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.964451 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 01:54:55.964640 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.964625 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 01:54:55.964716 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.964682 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\"" Nov 04 01:54:55.964770 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.964753 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 01:54:55.966945 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.966882 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 01:54:55.967035 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.966953 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 01:54:55.967035 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.966977 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 01:54:55.967035 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.967000 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 01:54:55.967183 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.967061 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\"" Nov 04 01:54:55.972821 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972804 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.972886 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s8vbd\" (UniqueName: \"kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.972886 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.972886 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972861 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.972886 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972879 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973006 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972915 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:55.973006 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.972943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973063 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973002 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973063 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:55.973145 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973070 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973145 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973094 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973145 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973145 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973126 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2vxdx\" (UniqueName: \"kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:55.973145 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973186 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973248 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973279 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bs62d\" (UniqueName: \"kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973327 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973370 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973389 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973417 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973411 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hgnnc\" (UniqueName: \"kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973454 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973504 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h2vnh\" (UniqueName: \"kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973551 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973591 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973605 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973605 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973637 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973664 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973704 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973721 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973734 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwbcd\" (UniqueName: \"kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973810 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqb6x\" (UniqueName: \"kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973859 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973878 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973894 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973910 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.973949 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973949 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973966 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.973982 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974072 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974100 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974131 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974183 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974200 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:55.974396 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974283 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974309 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974330 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974353 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974382 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jksbb\" (UniqueName: \"kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-skv9p\" (UniqueName: \"kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974441 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:55.974841 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.974455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:55.998735 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.998716 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 01:54:55.998735 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.998716 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\"" Nov 04 01:54:55.999173 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.998970 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\"" Nov 04 01:54:55.999173 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.999142 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\"" Nov 04 01:54:55.999173 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.999158 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 01:54:55.999393 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.999202 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\"" Nov 04 01:54:55.999393 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.999240 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 01:54:55.999393 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:55.999291 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 01:54:56.001124 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.001102 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 01:54:56.001124 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.001119 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 01:54:56.001269 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.001155 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 01:54:56.001269 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.001188 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 01:54:56.003607 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.003590 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 01:54:56.003607 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.003594 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 01:54:56.003739 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.003630 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 01:54:56.003739 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.003653 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 01:54:56.003739 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.003690 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 01:54:56.053256 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.053235 2580 reflector.go:446] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Nov 04 01:54:56.053341 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.053269 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-gp79h" Nov 04 01:54:56.053928 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.053906 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 01:54:56.075454 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.075537 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8vbd\" (UniqueName: \"kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.075537 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.075537 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075510 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.075537 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075529 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075574 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075639 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.075664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075653 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075666 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075681 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075714 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075718 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075691 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075735 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075739 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075769 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075769 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxdx\" (UniqueName: \"kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075833 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs62d\" (UniqueName: \"kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075888 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076073 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075911 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075945 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075948 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075981 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.075969 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076000 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076027 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgnnc\" (UniqueName: \"kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076072 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076078 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076113 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076133 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076154 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vnh\" (UniqueName: \"kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.076954 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076196 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076202 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076201 2580 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076219 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076242 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076263 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076284 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076430 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076522 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbcd\" (UniqueName: \"kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:56.077759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076526 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076525 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076538 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076556 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076538 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076545 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076592 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076630 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076634 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076690 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076645 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076726 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqb6x\" (UniqueName: \"kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076796 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.078412 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076813 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076905 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076915 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076959 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076965 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.076963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077009 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077056 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077064 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077066 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.079207 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077100 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077120 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077127 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077154 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077196 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077202 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077205 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077225 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077277 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077300 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077353 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077428 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.079771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077439 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077463 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077507 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077521 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077553 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077565 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077604 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.077619 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077658 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.077712 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:56.577671067 +0000 UTC m=+2.277770188 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077732 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077760 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077793 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jksbb\" (UniqueName: \"kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.080436 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077804 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077554 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077840 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skv9p\" (UniqueName: \"kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077862 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.077888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.077981 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.078149 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:56.578017976 +0000 UTC m=+2.278117073 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078412 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078659 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078808 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078819 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078869 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.078988 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.079206 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.079767 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.080916 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.079798 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.081356 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.080052 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.081356 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.080283 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:56.135564 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.135501 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:54:56.135564 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.135531 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:54:56.135564 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.135543 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:56.135776 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.135604 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:56.635586624 +0000 UTC m=+2.335685745 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:56.137671 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137016 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxdx\" (UniqueName: \"kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.137671 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137042 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8vbd\" (UniqueName: \"kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.137671 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137455 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqb6x\" (UniqueName: \"kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.137671 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137577 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbcd\" (UniqueName: \"kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:56.137860 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vnh\" (UniqueName: \"kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.137938 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.137914 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgnnc\" (UniqueName: \"kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.138191 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.138177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs62d\" (UniqueName: \"kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.143030 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.143005 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skv9p\" (UniqueName: \"kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.143105 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.143082 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jksbb\" (UniqueName: \"kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.238197 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.238081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:54:56.254827 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.254803 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:54:56.274874 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.274851 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:54:56.286883 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.286864 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:54:56.295727 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.294825 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod447db221_9de8_4e44_b228_ab76255a229d.slice/crio-ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222 WatchSource:0}: Error finding container ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222: Status 404 returned error can't find the container with id ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222 Nov 04 01:54:56.295727 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.295646 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c019a8a489976b5a638075ae287a9c6.slice/crio-1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e WatchSource:0}: Error finding container 1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e: Status 404 returned error can't find the container with id 1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e Nov 04 01:54:56.296536 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.296512 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a827277_7c60_4dbd_b5a5_fc861a2c236d.slice/crio-cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d WatchSource:0}: Error finding container cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d: Status 404 returned error can't find the container with id cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d Nov 04 01:54:56.302974 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.302956 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 01:54:56.303928 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.303905 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc65ed1_fdd2_4112_bc05_a24ec76ec6b0.slice/crio-390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6 WatchSource:0}: Error finding container 390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6: Status 404 returned error can't find the container with id 390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6 Nov 04 01:54:56.310562 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.310537 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:54:56.315448 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.315431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zxf47" Nov 04 01:54:56.326564 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.326526 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22ce2354_ad7a_4100_970f_ce84512cce41.slice/crio-de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c WatchSource:0}: Error finding container de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c: Status 404 returned error can't find the container with id de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c Nov 04 01:54:56.327126 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.327106 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podece54c3b_fb1f_4f03_9430_22f897a99c91.slice/crio-68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa WatchSource:0}: Error finding container 68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa: Status 404 returned error can't find the container with id 68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa Nov 04 01:54:56.327863 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.327803 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7db4f44b8ad055392b513e6afd9c7331.slice/crio-6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5 WatchSource:0}: Error finding container 6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5: Status 404 returned error can't find the container with id 6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5 Nov 04 01:54:56.329849 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.329822 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:54:56.339755 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.339732 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3e06791_0981_4894_9ac1_d55aec5adf00.slice/crio-fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1 WatchSource:0}: Error finding container fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1: Status 404 returned error can't find the container with id fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1 Nov 04 01:54:56.368730 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.368709 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:54:56.374598 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.374582 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:54:56.378803 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.378774 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0735e53_cdcf_4cb6_b68c_10ed8eb1cd60.slice/crio-2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e WatchSource:0}: Error finding container 2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e: Status 404 returned error can't find the container with id 2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e Nov 04 01:54:56.383519 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:56.383497 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0829f7ce_7f76_4727_bb67_7f60f08a3189.slice/crio-5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d WatchSource:0}: Error finding container 5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d: Status 404 returned error can't find the container with id 5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d Nov 04 01:54:56.581918 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.581883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:56.581918 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.581921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:56.582153 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.582031 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:56.582153 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.582033 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:56.582153 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.582086 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.582072114 +0000 UTC m=+3.282171227 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:56.582153 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.582111 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.582100968 +0000 UTC m=+3.282200065 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:56.682566 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.682453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:56.683285 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.682619 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:54:56.683285 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.682637 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:54:56.683285 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.682647 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:56.683285 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:56.683125 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:57.683112009 +0000 UTC m=+3.383211108 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:56.748559 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.748502 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"3c019a8a489976b5a638075ae287a9c6","Type":"ContainerStarted","Data":"1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e"} Nov 04 01:54:56.749718 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.749691 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d"} Nov 04 01:54:56.750763 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.750742 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zd9fs" event={"ID":"a3e06791-0981-4894-9ac1-d55aec5adf00","Type":"ContainerStarted","Data":"fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1"} Nov 04 01:54:56.751749 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.751729 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerStarted","Data":"de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c"} Nov 04 01:54:56.752763 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.752746 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdgnv" event={"ID":"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0","Type":"ContainerStarted","Data":"390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6"} Nov 04 01:54:56.753798 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.753781 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbhp4" event={"ID":"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60","Type":"ContainerStarted","Data":"2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e"} Nov 04 01:54:56.754799 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.754780 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerStarted","Data":"6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5"} Nov 04 01:54:56.755888 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.755869 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxf47" event={"ID":"ece54c3b-fb1f-4f03-9430-22f897a99c91","Type":"ContainerStarted","Data":"68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa"} Nov 04 01:54:56.756897 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.756880 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" event={"ID":"7a827277-7c60-4dbd-b5a5-fc861a2c236d","Type":"ContainerStarted","Data":"cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d"} Nov 04 01:54:56.757951 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.757931 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-75hc6" event={"ID":"447db221-9de8-4e44-b228-ab76255a229d","Type":"ContainerStarted","Data":"ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222"} Nov 04 01:54:56.843404 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:56.843374 2580 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:57.026826 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.026741 2580 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:57.056689 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.056637 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:56 +0000 UTC" deadline="2028-05-21 02:01:50.524468002 +0000 UTC" Nov 04 01:54:57.056689 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.056683 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13536h6m53.467788969s" Nov 04 01:54:57.081441 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:54:57.081413 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod078d2895_52e5_4a17_943a_ab3de7a3e40e.slice/crio-f748c59363bc35e83569c0f8719ac716f899f30b47adf35cb9ba6ae61501b9ac WatchSource:0}: Error finding container f748c59363bc35e83569c0f8719ac716f899f30b47adf35cb9ba6ae61501b9ac: Status 404 returned error can't find the container with id f748c59363bc35e83569c0f8719ac716f899f30b47adf35cb9ba6ae61501b9ac Nov 04 01:54:57.118069 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.118048 2580 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:54:57.591664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.591402 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:57.591664 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.591454 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:57.591664 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.591615 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:57.591664 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.591677 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:59.591658895 +0000 UTC m=+5.291757995 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:57.592052 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.591745 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:57.592052 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.591780 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:59.591768845 +0000 UTC m=+5.291867951 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:57.692227 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.691933 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:57.692227 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.692096 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:54:57.692227 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.692117 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:54:57.692227 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.692131 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:57.692227 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.692189 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:54:59.692172243 +0000 UTC m=+5.392271346 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:57.745176 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.745097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:57.745373 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.745232 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:54:57.745373 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.745321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:57.745500 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.745402 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:54:57.745500 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.745465 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:57.745634 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:57.745574 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:54:57.819576 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:57.814860 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" event={"ID":"078d2895-52e5-4a17-943a-ab3de7a3e40e","Type":"ContainerStarted","Data":"f748c59363bc35e83569c0f8719ac716f899f30b47adf35cb9ba6ae61501b9ac"} Nov 04 01:54:58.057250 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:58.057150 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:56 +0000 UTC" deadline="2028-05-20 22:25:35.310357234 +0000 UTC" Nov 04 01:54:58.057250 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:58.057198 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13532h30m37.253164613s" Nov 04 01:54:59.607865 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.607828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:59.608591 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.607940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:59.608591 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.608078 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:59.608591 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.608141 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:03.60812176 +0000 UTC m=+9.308221086 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:54:59.608591 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.608220 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:59.608591 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.608258 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:03.608245554 +0000 UTC m=+9.308344668 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:54:59.709357 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.709064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:59.709357 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.709243 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:54:59.709357 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.709263 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:54:59.709357 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.709272 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:59.709357 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.709324 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:03.709311373 +0000 UTC m=+9.409410472 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:54:59.747167 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.747119 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:54:59.747167 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.747142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:54:59.747316 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.747251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:54:59.747316 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:54:59.747258 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:54:59.747470 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.747401 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:54:59.747550 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:54:59.747514 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:00.840580 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:00.840269 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerStarted","Data":"44f591229c5e114b9f62e2e99195cf370f34d25675613c447b87f4f0abb09138"} Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.744749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:01.745116 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.745234 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:01.745342 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.745404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:01.745513 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:01.745483 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:01.853442 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.853381 2580 generic.go:356] "Generic (PLEG): container finished" podID="7db4f44b8ad055392b513e6afd9c7331" containerID="e2d379f9629553af3b15b4a46f8869e263ace0e89639392d81b77847e22c26ea" exitCode=0 Nov 04 01:55:01.853853 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.853527 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerDied","Data":"e2d379f9629553af3b15b4a46f8869e263ace0e89639392d81b77847e22c26ea"} Nov 04 01:55:01.871061 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.870965 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-75hc6" event={"ID":"447db221-9de8-4e44-b228-ab76255a229d","Type":"ContainerStarted","Data":"ed7b86459dde7ef8f5dec768554aca61c44acfe4d24c94b1c720c4897947209a"} Nov 04 01:55:01.882024 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.881386 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="44f591229c5e114b9f62e2e99195cf370f34d25675613c447b87f4f0abb09138" exitCode=0 Nov 04 01:55:01.882024 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:01.881435 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"44f591229c5e114b9f62e2e99195cf370f34d25675613c447b87f4f0abb09138"} Nov 04 01:55:02.236698 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:02.236136 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/konnectivity-agent-75hc6" podStartSLOduration=1.955738968 podStartE2EDuration="7.236114973s" totalImagesPullingTime="5.280376005s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:02.10368059 +0000 UTC m=+7.803779738" watchObservedRunningTime="2026-11-04 01:55:02.236114973 +0000 UTC m=+7.936214162" Nov 04 01:55:02.893221 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:02.893133 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-mbhp4" event={"ID":"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60","Type":"ContainerStarted","Data":"61a50abbbfb37513faed06ba9b6dc16ca0af886e53a3c5bccb72256e447dbf9c"} Nov 04 01:55:02.897427 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:02.897397 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerStarted","Data":"f971cac9e265263b800f353ab662dc729b391a58d319ca3826137039fbb26827"} Nov 04 01:55:02.900337 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:02.900299 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"3c019a8a489976b5a638075ae287a9c6","Type":"ContainerStarted","Data":"0211f3fe3fa0b4810003ece67154aefba6dfaf87ae8f295d9487adfc0beac5a6"} Nov 04 01:55:03.141735 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.141670 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/node-ca-mbhp4" podStartSLOduration=2.039012525 podStartE2EDuration="8.141649241s" totalImagesPullingTime="6.102636716s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:03.018518063 +0000 UTC m=+8.718617187" watchObservedRunningTime="2026-11-04 01:55:03.141649241 +0000 UTC m=+8.841748364" Nov 04 01:55:03.267140 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.267069 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" podStartSLOduration=8.267048421 podStartE2EDuration="8.267048421s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:03.142515398 +0000 UTC m=+8.842614527" watchObservedRunningTime="2026-11-04 01:55:03.267048421 +0000 UTC m=+8.967147546" Nov 04 01:55:03.267297 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.267235 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" podStartSLOduration=7.267225554 podStartE2EDuration="7.267225554s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:03.266658767 +0000 UTC m=+8.966757895" watchObservedRunningTime="2026-11-04 01:55:03.267225554 +0000 UTC m=+8.967324677" Nov 04 01:55:03.646086 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.646028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:03.646086 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.646088 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:03.646349 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.646210 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:03.646349 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.646292 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:11.646269049 +0000 UTC m=+17.346368154 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:03.646349 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.646210 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:03.646349 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.646344 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:11.646334843 +0000 UTC m=+17.346433945 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:03.745639 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.745602 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:03.745818 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.745740 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:03.745818 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.745753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:03.745966 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.745879 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:03.745966 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.745956 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:03.746087 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.746051 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:03.746749 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.746727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:03.746901 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.746878 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:55:03.746966 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.746912 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:55:03.746966 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.746925 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:03.747072 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:03.746976 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:11.746958565 +0000 UTC m=+17.447057680 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:03.905316 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:03.905236 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" event={"ID":"7a827277-7c60-4dbd-b5a5-fc861a2c236d","Type":"ContainerStarted","Data":"83e8ff1fd7f473e110b7faf0a9fd20f15cec4caf3e2ce58f0285e18a6c85c1f9"} Nov 04 01:55:05.049837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.049802 2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:55:05.050973 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.050948 2580 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:55:05.736503 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.736447 2580 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.744652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.744653 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:05.744747 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.744760 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:05.744836 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:05.747087 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:05.744912 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:05.754811 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.754713 2580 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:55:05.736474528Z","UUID":"9be88948-36d3-4c7d-95a6-e9df4dd36417","Handler":null,"Name":"","Endpoint":""} Nov 04 01:55:05.756688 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.756667 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 01:55:05.756789 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.756769 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 01:55:05.912022 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.911985 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-vdgnv" event={"ID":"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0","Type":"ContainerStarted","Data":"54cb46ec8293a0bea8bff6ef7efc421a5122cfa672bacb25f77a89a938b8746a"} Nov 04 01:55:05.914790 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.914760 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zxf47" event={"ID":"ece54c3b-fb1f-4f03-9430-22f897a99c91","Type":"ContainerStarted","Data":"1407ea7a83128abf3db8b13fde5a8a6d739924e22eb485993cb32777ab775a87"} Nov 04 01:55:05.918282 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.918247 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" event={"ID":"7a827277-7c60-4dbd-b5a5-fc861a2c236d","Type":"ContainerStarted","Data":"800bbcf39a9a87368e20a89f1a123c0970a833b82a862fe0b366c98b48195bf0"} Nov 04 01:55:05.918402 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.918327 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:55:05.918853 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:05.918819 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:55:06.037837 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:06.037780 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/node-resolver-vdgnv" podStartSLOduration=1.691008746 podStartE2EDuration="11.037762183s" totalImagesPullingTime="9.346753437s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:06.036933605 +0000 UTC m=+11.737032925" watchObservedRunningTime="2026-11-04 01:55:06.037762183 +0000 UTC m=+11.737861310" Nov 04 01:55:06.287955 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:06.287834 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-zxf47" podStartSLOduration=2.189653492 podStartE2EDuration="11.287818338s" totalImagesPullingTime="9.098164846s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:06.163973306 +0000 UTC m=+11.864072438" watchObservedRunningTime="2026-11-04 01:55:06.287818338 +0000 UTC m=+11.987917481" Nov 04 01:55:06.923280 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:06.922813 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="d8f5ad2f6616a3141ec7ed4292613fe2c3cb6630a02758dc0513dceab4901f41" exitCode=0 Nov 04 01:55:06.923665 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:06.923521 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"d8f5ad2f6616a3141ec7ed4292613fe2c3cb6630a02758dc0513dceab4901f41"} Nov 04 01:55:07.744844 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:07.744598 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:07.744844 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:07.744639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:07.744844 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:07.744639 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:07.744844 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:07.744748 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:07.745404 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:07.744844 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:07.745404 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:07.744952 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:07.926839 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:07.926803 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-zd9fs" event={"ID":"a3e06791-0981-4894-9ac1-d55aec5adf00","Type":"ContainerStarted","Data":"c87790fbcc7fab243ab62fb9edec927266e679ac89d478daee264373c18eef30"} Nov 04 01:55:08.040497 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:08.040436 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-zd9fs" podStartSLOduration=3.368829758 podStartE2EDuration="13.040423021s" totalImagesPullingTime="9.671593263s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:08.040273711 +0000 UTC m=+13.740372848" watchObservedRunningTime="2026-11-04 01:55:08.040423021 +0000 UTC m=+13.740522140" Nov 04 01:55:08.930930 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:08.930846 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" event={"ID":"7a827277-7c60-4dbd-b5a5-fc861a2c236d","Type":"ContainerStarted","Data":"6b6d7b87afce05b61a509bbe946326a5c4f3aa0d1509084ad398266b220543e8"} Nov 04 01:55:08.933525 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:08.933501 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="00fad87fe5bbdcb0e08d0eba319989838a3dd094ebdabec80cb33b932a7762db" exitCode=0 Nov 04 01:55:08.933653 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:08.933585 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"00fad87fe5bbdcb0e08d0eba319989838a3dd094ebdabec80cb33b932a7762db"} Nov 04 01:55:08.935254 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:08.935232 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" event={"ID":"078d2895-52e5-4a17-943a-ab3de7a3e40e","Type":"ContainerStarted","Data":"d21e2ea0c7d640e8d112029878526168f431b0dd0136bf050e25f373e7723601"} Nov 04 01:55:09.047793 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:09.047739 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" podStartSLOduration=2.749355052 podStartE2EDuration="14.04772063s" totalImagesPullingTime="11.298365578s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:09.047080038 +0000 UTC m=+14.747179179" watchObservedRunningTime="2026-11-04 01:55:09.04772063 +0000 UTC m=+14.747819752" Nov 04 01:55:09.745139 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:09.745109 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:09.745291 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:09.745197 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:09.745291 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:09.745281 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:09.745410 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:09.745301 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:09.745410 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:09.745382 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:09.745536 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:09.745509 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:10.942212 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:10.942104 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="51a18529b960f26bf6f43505ece722f45d84b1ac4133ba05d3c7f7840a485c74" exitCode=0 Nov 04 01:55:10.942212 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:10.942170 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"51a18529b960f26bf6f43505ece722f45d84b1ac4133ba05d3c7f7840a485c74"} Nov 04 01:55:11.064035 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.063976 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" podStartSLOduration=4.87033293 podStartE2EDuration="16.063958995s" totalImagesPullingTime="11.193626065s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:09.285793949 +0000 UTC m=+14.985893067" watchObservedRunningTime="2026-11-04 01:55:11.063958995 +0000 UTC m=+16.764058115" Nov 04 01:55:11.714103 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.713882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:11.714292 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.714126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:11.714292 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.714031 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:11.714402 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.714299 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:27.714285723 +0000 UTC m=+33.414384821 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:11.714402 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.714247 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:11.714402 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.714330 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:27.71432362 +0000 UTC m=+33.414422716 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:11.745489 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.745447 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:11.745645 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.745595 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:11.745723 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.745704 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:11.745766 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.745593 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:11.745819 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.745592 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:11.745872 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.745826 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:11.815125 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:11.815084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:11.815292 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.815247 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:55:11.815292 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.815270 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:55:11.815292 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.815280 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:11.815426 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:11.815335 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:27.815318459 +0000 UTC m=+33.515417558 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:13.744939 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.744910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:13.744939 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.744936 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:13.745384 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.744945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:13.745384 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:13.745056 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:13.745384 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:13.745140 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:13.745384 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:13.745206 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:13.952371 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.952341 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"7bfe5f611b9d3594ab8d99ff3c79889a03929810a6d29142173f873a0f8d578f"} Nov 04 01:55:13.952505 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.952435 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"77e1d07fc528aab062651ac6992041ed71bd22d26aeebf7ff2478eb280caa0ba"} Nov 04 01:55:13.955455 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.955429 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="af8d03a5719bfa9d86f95214a9ff558fc701206b063a0203547ca331e09b0489" exitCode=0 Nov 04 01:55:13.955582 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:13.955491 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"af8d03a5719bfa9d86f95214a9ff558fc701206b063a0203547ca331e09b0489"} Nov 04 01:55:14.961151 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.960917 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"0d40c88ad8ab55c5fd395c6a046d9905e4961a4c3207fa6deabc036728ebcaec"} Nov 04 01:55:14.961602 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.961217 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"070d29d637f499e5f28128e75f2ad5096b097c829bec9635866ed5fc4ca168c1"} Nov 04 01:55:14.961602 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.961262 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"1f385d9932d0b3a22fd304bc7b71fe0230b730a216a1d9b074ee4b40b509ec22"} Nov 04 01:55:14.961602 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.961295 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"9c940b0f5edd43f54414203b53ecb31908f8395e9364c5805c63a4685e8169e8"} Nov 04 01:55:14.963958 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.963934 2580 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="9da1ceb6b63df8a94678da5bf44f124873835f435ba89b100b3e9878289d9729" exitCode=0 Nov 04 01:55:14.964070 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:14.963966 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerDied","Data":"9da1ceb6b63df8a94678da5bf44f124873835f435ba89b100b3e9878289d9729"} Nov 04 01:55:15.745100 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:15.745070 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:15.745268 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:15.745141 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:15.745268 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:15.745164 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:15.745333 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:15.745265 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:15.745333 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:15.745299 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:15.745398 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:15.745352 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:15.969778 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:15.969745 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-59clt" event={"ID":"22ce2354-ad7a-4100-970f-ce84512cce41","Type":"ContainerStarted","Data":"906b57b425dbd253fb14020615f587b606723e52a3597abdad391902c890c742"} Nov 04 01:55:16.090905 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:16.090823 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-59clt" podStartSLOduration=7.432434488 podStartE2EDuration="21.090810207s" totalImagesPullingTime="13.658375719s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:16.090380921 +0000 UTC m=+21.790480042" watchObservedRunningTime="2026-11-04 01:55:16.090810207 +0000 UTC m=+21.790909326" Nov 04 01:55:16.975367 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:16.975335 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"1ea1287200bc8716f263ac67c04b65262457bf9a3f3edbc9f354f74939e74fcb"} Nov 04 01:55:17.745082 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:17.745048 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:17.745082 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:17.745076 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:17.745428 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:17.745161 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:17.745428 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:17.745189 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:17.745428 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:17.745251 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:17.745428 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:17.745325 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:18.983235 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:18.983197 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" event={"ID":"0829f7ce-7f76-4727-bb67-7f60f08a3189","Type":"ContainerStarted","Data":"cec3507a91ae91ce94dbca58a57fb449bf0ee6b52456093434adeddf6fa1544e"} Nov 04 01:55:18.984023 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:18.983445 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:18.984023 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:18.983530 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:18.999406 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:18.999380 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:18.999567 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:18.999555 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:19.259342 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:19.259254 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" podStartSLOduration=6.877873276 podStartE2EDuration="24.259242214s" totalImagesPullingTime="17.381368938s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:19.235131898 +0000 UTC m=+24.935231017" watchObservedRunningTime="2026-11-04 01:55:19.259242214 +0000 UTC m=+24.959341333" Nov 04 01:55:19.745539 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:19.745434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:19.745539 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:19.745453 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:19.745539 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:19.745505 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:19.745789 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:19.745576 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:19.745789 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:19.745633 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:19.745789 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:19.745736 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:19.985187 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:19.985161 2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 01:55:20.975778 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:20.975507 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sd84m"] Nov 04 01:55:20.975943 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:20.975803 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:20.975943 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:20.975901 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:20.987672 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:20.987648 2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 01:55:21.024584 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:21.024564 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w2257"] Nov 04 01:55:21.024704 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:21.024659 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:21.024749 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:21.024732 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:21.025033 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:21.025017 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkzgd"] Nov 04 01:55:21.025104 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:21.025095 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:21.025170 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:21.025155 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:22.744855 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:22.744790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:22.745283 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:22.744790 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:22.745283 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:22.744933 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:22.745283 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:22.744791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:22.745283 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:22.744965 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:22.745283 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:22.745035 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:23.493460 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:23.493376 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:23.493630 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:23.493595 2580 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 01:55:23.504851 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:23.504808 2580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" podUID="0829f7ce-7f76-4727-bb67-7f60f08a3189" containerName="ovnkube-controller" probeResult="failure" output="" Nov 04 01:55:23.513877 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:23.513851 2580 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" podUID="0829f7ce-7f76-4727-bb67-7f60f08a3189" containerName="ovnkube-controller" probeResult="failure" output="" Nov 04 01:55:24.746488 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:24.746443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:24.746964 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:24.746557 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:24.746964 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:24.746588 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:24.746964 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:24.746606 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:24.746964 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:24.746661 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:24.746964 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:24.746745 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:26.745658 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:26.745580 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:26.745658 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:26.745604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:26.746029 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:26.745682 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkzgd" podUID="636d2d6a-86f9-41ad-95f8-2b79b2645544" Nov 04 01:55:26.746029 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:26.745744 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:26.746029 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:26.745841 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-w2257" podUID="38e16a3b-bcac-4482-b177-eb0061845a74" Nov 04 01:55:26.746029 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:26.745937 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-sd84m" podUID="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" Nov 04 01:55:27.631734 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.631665 2580 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeReady" Nov 04 01:55:27.631867 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.631765 2580 kubelet_node_status.go:477] "Fast updating node status as it just became ready" Nov 04 01:55:27.725660 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.725626 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:27.725813 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.725707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:27.725813 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.725778 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:27.725882 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.725812 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:27.725882 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.725835 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs podName:a9dfb0df-5635-4cbd-81e6-6c55f20b3a17 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:59.725821141 +0000 UTC m=+65.425920238 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs") pod "network-metrics-daemon-sd84m" (UID: "a9dfb0df-5635-4cbd-81e6-6c55f20b3a17") : object "openshift-multus"/"metrics-daemon-secret" not registered Nov 04 01:55:27.725882 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.725865 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret podName:38e16a3b-bcac-4482-b177-eb0061845a74 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:59.725852443 +0000 UTC m=+65.425951540 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret") pod "global-pull-secret-syncer-w2257" (UID: "38e16a3b-bcac-4482-b177-eb0061845a74") : object "kube-system"/"original-pull-secret" not registered Nov 04 01:55:27.785905 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.785878 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p9lv2"] Nov 04 01:55:27.820585 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.820558 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-rjstr"] Nov 04 01:55:27.820704 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.820675 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:27.826109 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.826084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:27.826233 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.826220 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Nov 04 01:55:27.826280 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.826239 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Nov 04 01:55:27.826280 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.826248 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7xdjn for pod openshift-network-diagnostics/network-check-target-nkzgd: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:27.826339 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:27.826294 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn podName:636d2d6a-86f9-41ad-95f8-2b79b2645544 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:59.826279391 +0000 UTC m=+65.526378490 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7xdjn" (UniqueName: "kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn") pod "network-check-target-nkzgd" (UID: "636d2d6a-86f9-41ad-95f8-2b79b2645544") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Nov 04 01:55:27.838165 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.838143 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:27.859805 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.859779 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjstr"] Nov 04 01:55:27.863492 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.863454 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p9lv2"] Nov 04 01:55:27.873362 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.873342 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 01:55:27.881630 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.881612 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 01:55:27.881847 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.881817 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\"" Nov 04 01:55:27.890003 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.889984 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 01:55:27.890114 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.890033 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 01:55:27.890163 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.890143 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 01:55:27.890213 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.890194 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\"" Nov 04 01:55:27.927104 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6qgt4\" (UniqueName: \"kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:27.927194 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:27.927194 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:27.927194 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:27.927349 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927253 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kdrmc\" (UniqueName: \"kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:27.927349 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:27.927326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.027644 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027616 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:28.027784 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027667 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrmc\" (UniqueName: \"kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.027784 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027719 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.027784 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027746 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgt4\" (UniqueName: \"kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:28.027784 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.027955 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.027909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.028030 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:28.028012 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 01:55:28.028085 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:28.028076 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls podName:0a1d99ff-e629-46da-a1e0-e92fd128a237 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:28.52805832 +0000 UTC m=+34.228157419 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls") pod "dns-default-p9lv2" (UID: "0a1d99ff-e629-46da-a1e0-e92fd128a237") : secret "dns-default-metrics-tls" not found Nov 04 01:55:28.028142 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.028107 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.028250 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.028224 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.031638 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.031614 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:28.088691 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.088661 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrmc\" (UniqueName: \"kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.088800 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.088777 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgt4\" (UniqueName: \"kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:28.178988 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.178916 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:55:28.532284 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.532215 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:28.532449 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:28.532391 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 01:55:28.532537 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:28.532496 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls podName:0a1d99ff-e629-46da-a1e0-e92fd128a237 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:29.532451899 +0000 UTC m=+35.232550999 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls") pod "dns-default-p9lv2" (UID: "0a1d99ff-e629-46da-a1e0-e92fd128a237") : secret "dns-default-metrics-tls" not found Nov 04 01:55:28.616360 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.616330 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-rjstr"] Nov 04 01:55:28.619484 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:55:28.619442 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf456ccdc_f27b_4026_b425_891f5c46263e.slice/crio-ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6 WatchSource:0}: Error finding container ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6: Status 404 returned error can't find the container with id ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6 Nov 04 01:55:28.744963 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.744923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:28.744963 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.744961 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:28.745133 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.744966 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:28.797371 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.797290 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 01:55:28.797742 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.797575 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\"" Nov 04 01:55:28.799983 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.799966 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\"" Nov 04 01:55:28.800114 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.800098 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\"" Nov 04 01:55:28.800172 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.800122 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 01:55:28.800172 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.800135 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 01:55:28.800251 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:28.800171 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 01:55:29.006196 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.006162 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjstr" event={"ID":"f456ccdc-f27b-4026-b425-891f5c46263e","Type":"ContainerStarted","Data":"ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6"} Nov 04 01:55:29.538651 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.538615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:29.538824 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.538721 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 01:55:29.538824 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.538791 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls podName:0a1d99ff-e629-46da-a1e0-e92fd128a237 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.538771217 +0000 UTC m=+37.238870315 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls") pod "dns-default-p9lv2" (UID: "0a1d99ff-e629-46da-a1e0-e92fd128a237") : secret "dns-default-metrics-tls" not found Nov 04 01:55:29.660765 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.660738 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-mrgm4"] Nov 04 01:55:29.675603 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.675573 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.722788 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.722754 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mrgm4"] Nov 04 01:55:29.740755 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.740733 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.740866 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.740767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.740866 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.740790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbqdz\" (UniqueName: \"kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.740992 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.740932 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.740992 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.740967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.814211 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.814125 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 01:55:29.814211 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.814122 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 01:55:29.814211 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.814159 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 01:55:29.814211 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.814186 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\"" Nov 04 01:55:29.814660 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.814236 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 01:55:29.841614 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqdz\" (UniqueName: \"kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.841744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.841744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841674 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.841744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841698 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.841744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841727 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.841920 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.841814 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 01:55:29.841920 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.841867 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.341852116 +0000 UTC m=+36.041951230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : configmap "kube-rbac-proxy" not found Nov 04 01:55:29.841920 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.841878 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 01:55:29.842051 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:29.841932 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:30.341916791 +0000 UTC m=+36.042015893 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : secret "insights-runtime-extractor-tls" not found Nov 04 01:55:29.842051 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.841940 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.842149 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.842075 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:29.903034 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:29.903007 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqdz\" (UniqueName: \"kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:30.345980 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:30.345941 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:30.345980 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:30.345995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:30.346397 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:30.346087 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 01:55:30.346397 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:30.346143 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 01:55:30.346397 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:30.346170 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.346151696 +0000 UTC m=+37.046250814 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : configmap "kube-rbac-proxy" not found Nov 04 01:55:30.346397 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:30.346197 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:31.346182838 +0000 UTC m=+37.046281936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : secret "insights-runtime-extractor-tls" not found Nov 04 01:55:31.353420 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.353376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:31.353420 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.353425 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:31.353930 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.353550 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 01:55:31.353930 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.353557 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 01:55:31.353930 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.353629 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.353608913 +0000 UTC m=+39.053708025 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : configmap "kube-rbac-proxy" not found Nov 04 01:55:31.353930 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.353671 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:33.353650952 +0000 UTC m=+39.053750066 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : secret "insights-runtime-extractor-tls" not found Nov 04 01:55:31.362537 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.362511 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj"] Nov 04 01:55:31.394505 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.394462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:55:31.422519 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.422461 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj"] Nov 04 01:55:31.454307 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.454274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8647w\" (UniqueName: \"kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w\") pod \"migrator-66df5d496b-zdttj\" (UID: \"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:55:31.530790 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.530752 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 01:55:31.530952 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.530860 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 01:55:31.531013 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.530965 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c7hth\"" Nov 04 01:55:31.555323 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.555285 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8647w\" (UniqueName: \"kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w\") pod \"migrator-66df5d496b-zdttj\" (UID: \"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:55:31.555323 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.555322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:31.555548 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.555441 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 01:55:31.555548 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:31.555504 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls podName:0a1d99ff-e629-46da-a1e0-e92fd128a237 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:35.555492574 +0000 UTC m=+41.255591672 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls") pod "dns-default-p9lv2" (UID: "0a1d99ff-e629-46da-a1e0-e92fd128a237") : secret "dns-default-metrics-tls" not found Nov 04 01:55:31.616173 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.616103 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8647w\" (UniqueName: \"kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w\") pod \"migrator-66df5d496b-zdttj\" (UID: \"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:55:31.716998 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:31.716955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:55:32.153556 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:32.153534 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj"] Nov 04 01:55:33.017390 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:33.017358 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-rjstr" event={"ID":"f456ccdc-f27b-4026-b425-891f5c46263e","Type":"ContainerStarted","Data":"3c107019f542e1ad4324429c06e0e7be3597339d60b3fb2ddefe443df1aeb4e2"} Nov 04 01:55:33.018524 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:33.018501 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" event={"ID":"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f","Type":"ContainerStarted","Data":"099ab16ecfe60ce11d78c63be8796037b3e4f3052981cf895edf7cd696027225"} Nov 04 01:55:33.133378 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:33.133328 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-rjstr" podStartSLOduration=2.612458456 podStartE2EDuration="6.133313522s" totalImagesPullingTime="3.520855066s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:33.132771915 +0000 UTC m=+38.832871056" watchObservedRunningTime="2026-11-04 01:55:33.133313522 +0000 UTC m=+38.833412667" Nov 04 01:55:33.369413 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:33.369333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:33.369413 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:33.369379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:33.369703 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:33.369493 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Nov 04 01:55:33.369703 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:33.369517 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 01:55:33.369703 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:33.369562 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.369548357 +0000 UTC m=+43.069647454 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : configmap "kube-rbac-proxy" not found Nov 04 01:55:33.369703 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:33.369577 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:37.369571159 +0000 UTC m=+43.069670255 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : secret "insights-runtime-extractor-tls" not found Nov 04 01:55:35.024252 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:35.024214 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" event={"ID":"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f","Type":"ContainerStarted","Data":"0e0be184c62062a564f514b3dc0a7bba3b3875c52f491b994dd271e5ceb4e454"} Nov 04 01:55:35.139140 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:35.139088 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" podStartSLOduration=2.200412325 podStartE2EDuration="4.13907612s" totalImagesPullingTime="1.938663795s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:35.138253697 +0000 UTC m=+40.838352836" watchObservedRunningTime="2026-11-04 01:55:35.13907612 +0000 UTC m=+40.839175239" Nov 04 01:55:35.589665 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:35.589629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:35.589836 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:35.589728 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Nov 04 01:55:35.589836 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:35.589794 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls podName:0a1d99ff-e629-46da-a1e0-e92fd128a237 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:43.589777557 +0000 UTC m=+49.289876654 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls") pod "dns-default-p9lv2" (UID: "0a1d99ff-e629-46da-a1e0-e92fd128a237") : secret "dns-default-metrics-tls" not found Nov 04 01:55:37.402972 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:37.402937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:37.403356 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:37.402979 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:37.403356 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:37.403071 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Nov 04 01:55:37.403356 ip-10-0-134-176 kubenswrapper[2580]: E1104 01:55:37.403121 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls podName:bb2c7ac1-0806-4d8c-8c19-979d3c080f80 nodeName:}" failed. No retries permitted until 2026-11-04 01:55:45.403109064 +0000 UTC m=+51.103208161 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls") pod "insights-runtime-extractor-mrgm4" (UID: "bb2c7ac1-0806-4d8c-8c19-979d3c080f80") : secret "insights-runtime-extractor-tls" not found Nov 04 01:55:37.403490 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:37.403438 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:37.831519 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:37.831490 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-vdgnv_ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0/dns-node-resolver/0.log" Nov 04 01:55:38.634542 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:38.634454 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-75hc6_447db221-9de8-4e44-b228-ab76255a229d/konnectivity-agent/0.log" limit=1024 Nov 04 01:55:38.687128 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:38.687101 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-mbhp4_f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60/node-ca/0.log" Nov 04 01:55:38.908689 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:38.908608 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-75hc6_447db221-9de8-4e44-b228-ab76255a229d/konnectivity-agent/0.log" limit=1024 Nov 04 01:55:39.174727 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:39.174657 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-75hc6_447db221-9de8-4e44-b228-ab76255a229d/konnectivity-agent/0.log" limit=1024 Nov 04 01:55:39.631529 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:39.631507 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-rjstr_f456ccdc-f27b-4026-b425-891f5c46263e/serve-healthcheck-canary/0.log" Nov 04 01:55:40.031552 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:40.031526 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-66df5d496b-zdttj_7cc1dde7-35d0-4bec-8ead-b06ecff9f71f/migrator/0.log" Nov 04 01:55:43.644839 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:43.644803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:43.647217 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:43.647195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:43.757100 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:43.757059 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:44.125110 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:44.125078 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p9lv2"] Nov 04 01:55:44.128779 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:55:44.128746 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a1d99ff_e629_46da_a1e0_e92fd128a237.slice/crio-2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1 WatchSource:0}: Error finding container 2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1: Status 404 returned error can't find the container with id 2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1 Nov 04 01:55:45.047488 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:45.047441 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9lv2" event={"ID":"0a1d99ff-e629-46da-a1e0-e92fd128a237","Type":"ContainerStarted","Data":"2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1"} Nov 04 01:55:45.460719 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:45.460633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:45.463288 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:45.463266 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:45.626203 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:45.626167 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:55:45.996932 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:45.996900 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-mrgm4"] Nov 04 01:55:46.028235 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:55:46.028208 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbb2c7ac1_0806_4d8c_8c19_979d3c080f80.slice/crio-e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e WatchSource:0}: Error finding container e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e: Status 404 returned error can't find the container with id e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e Nov 04 01:55:46.050902 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:46.050869 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrgm4" event={"ID":"bb2c7ac1-0806-4d8c-8c19-979d3c080f80","Type":"ContainerStarted","Data":"e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e"} Nov 04 01:55:47.054831 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:47.054791 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrgm4" event={"ID":"bb2c7ac1-0806-4d8c-8c19-979d3c080f80","Type":"ContainerStarted","Data":"883bd90f0337c1ff700ebcf88f3e5f14596ff3c1bc82b272b98a8b7114e731b8"} Nov 04 01:55:47.056586 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:47.056563 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9lv2" event={"ID":"0a1d99ff-e629-46da-a1e0-e92fd128a237","Type":"ContainerStarted","Data":"1b44eaa09d7126d794c54288984fe3ab308c3e6ab5f837f369fae7a6949756d1"} Nov 04 01:55:47.056684 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:47.056606 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p9lv2" event={"ID":"0a1d99ff-e629-46da-a1e0-e92fd128a237","Type":"ContainerStarted","Data":"3a065bffa18d4cfe2306af20027526bc4d4aebb0dd54dcde981b16458d9e5f68"} Nov 04 01:55:47.056720 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:47.056704 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:47.174552 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:47.174493 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-dns/dns-default-p9lv2" podStartSLOduration=18.03937903 podStartE2EDuration="20.174459922s" totalImagesPullingTime="2.135080892s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:47.173925167 +0000 UTC m=+52.874024305" watchObservedRunningTime="2026-11-04 01:55:47.174459922 +0000 UTC m=+52.874559041" Nov 04 01:55:48.060613 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:48.060581 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrgm4" event={"ID":"bb2c7ac1-0806-4d8c-8c19-979d3c080f80","Type":"ContainerStarted","Data":"99783889a2512479282ee59b1f115947e95bc87f9d60a6d916bcfd50d6d95371"} Nov 04 01:55:50.067880 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:50.067845 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-mrgm4" event={"ID":"bb2c7ac1-0806-4d8c-8c19-979d3c080f80","Type":"ContainerStarted","Data":"b1cf4a349aecef59a1faf980983fbdcbd8fa5ad83b19760078097f99a8f3ba47"} Nov 04 01:55:50.185423 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:50.185374 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-mrgm4" podStartSLOduration=17.883291577 podStartE2EDuration="21.185360516s" totalImagesPullingTime="3.302068939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 01:55:29 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:50.184771932 +0000 UTC m=+55.884871072" watchObservedRunningTime="2026-11-04 01:55:50.185360516 +0000 UTC m=+55.885459634" Nov 04 01:55:52.395793 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.395759 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-75fb56bf5f-nt7k7"] Nov 04 01:55:52.397607 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.397592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.512791 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.512763 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.512944 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.512798 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.512944 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.512827 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.512944 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.512896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.513075 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.513062 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.513106 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.513082 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4tmtz\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.513106 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.513099 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.513160 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.513121 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.550686 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.550659 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 01:55:52.551744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.551725 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fb56bf5f-nt7k7"] Nov 04 01:55:52.583497 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.583436 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 01:55:52.583497 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.583460 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 01:55:52.583639 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.583515 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zsfkj\"" Nov 04 01:55:52.583639 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.583518 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 01:55:52.613639 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613617 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.613757 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tmtz\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.613757 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.613757 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613679 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.613757 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613715 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.614012 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613779 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.614012 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613817 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.614012 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.613842 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.614389 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.614366 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.615017 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.614939 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.615282 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.615265 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.616660 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.616641 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.616994 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.616968 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.616994 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.616975 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.674077 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.674023 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.674212 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.674195 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tmtz\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:52.707034 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:52.707010 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:53.174692 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:53.174658 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-75fb56bf5f-nt7k7"] Nov 04 01:55:53.212320 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:55:53.212294 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod840c4e7c_424c_4f46_8f30_cb39f2d31a24.slice/crio-94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f WatchSource:0}: Error finding container 94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f: Status 404 returned error can't find the container with id 94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f Nov 04 01:55:53.515203 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:53.515174 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:55:54.079834 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:54.079795 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" event={"ID":"840c4e7c-424c-4f46-8f30-cb39f2d31a24","Type":"ContainerStarted","Data":"244b0d3300bcdd2c86fea7708893029b1a01b4ba50a8172f85a23cb32bfd31f4"} Nov 04 01:55:54.080012 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:54.079847 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" event={"ID":"840c4e7c-424c-4f46-8f30-cb39f2d31a24","Type":"ContainerStarted","Data":"94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f"} Nov 04 01:55:54.080696 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:54.080274 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:55:54.201308 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:54.201262 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" podStartSLOduration=2.201247395 podStartE2EDuration="2.201247395s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:55:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:55:54.200333044 +0000 UTC m=+59.900432190" watchObservedRunningTime="2026-11-04 01:55:54.201247395 +0000 UTC m=+59.901346513" Nov 04 01:55:57.063411 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:57.063377 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p9lv2" Nov 04 01:55:59.767111 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.767080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:59.767560 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.767148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:59.867974 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.867938 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 01:55:59.868130 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.867991 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 01:55:59.868130 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.868074 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:55:59.880440 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.880416 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:55:59.880561 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.880445 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:55:59.919197 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.919172 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 01:55:59.980192 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.980169 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 01:55:59.991686 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:55:59.991665 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:00.024771 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.024723 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\"" Nov 04 01:56:00.033926 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.033909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:00.048799 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.048782 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\"" Nov 04 01:56:00.057875 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.057856 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:56:00.317238 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.317178 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\"" Nov 04 01:56:00.326282 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.326245 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:00.399385 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.399356 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-w2257"] Nov 04 01:56:00.442367 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:56:00.442340 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38e16a3b_bcac_4482_b177_eb0061845a74.slice/crio-9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f WatchSource:0}: Error finding container 9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f: Status 404 returned error can't find the container with id 9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f Nov 04 01:56:00.450005 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.449982 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-sd84m"] Nov 04 01:56:00.452890 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:56:00.452866 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda9dfb0df_5635_4cbd_81e6_6c55f20b3a17.slice/crio-00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77 WatchSource:0}: Error finding container 00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77: Status 404 returned error can't find the container with id 00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77 Nov 04 01:56:00.725697 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:00.725613 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkzgd"] Nov 04 01:56:00.729600 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:56:00.729572 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod636d2d6a_86f9_41ad_95f8_2b79b2645544.slice/crio-ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96 WatchSource:0}: Error finding container ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96: Status 404 returned error can't find the container with id ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96 Nov 04 01:56:01.101833 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:01.101798 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sd84m" event={"ID":"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17","Type":"ContainerStarted","Data":"00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77"} Nov 04 01:56:01.102990 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:01.102966 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-w2257" event={"ID":"38e16a3b-bcac-4482-b177-eb0061845a74","Type":"ContainerStarted","Data":"9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f"} Nov 04 01:56:01.104124 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:01.104104 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkzgd" event={"ID":"636d2d6a-86f9-41ad-95f8-2b79b2645544","Type":"ContainerStarted","Data":"ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96"} Nov 04 01:56:02.110699 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:02.110671 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sd84m" event={"ID":"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17","Type":"ContainerStarted","Data":"7e1120284ea8c551c0dfaad947503d2d160f7c8d7999696e895201dbc16d0dd6"} Nov 04 01:56:03.117076 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.117034 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-sd84m" event={"ID":"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17","Type":"ContainerStarted","Data":"15d9d3572a612dc44431b8691fb9baabfa2b3a21fe61a1d4cbadd0e9a80d63f6"} Nov 04 01:56:03.232669 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.232601 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-sd84m" podStartSLOduration=66.706598366 podStartE2EDuration="1m8.232583095s" totalImagesPullingTime="1.525984729s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:03.231190146 +0000 UTC m=+68.931289265" watchObservedRunningTime="2026-11-04 01:56:03.232583095 +0000 UTC m=+68.932682215" Nov 04 01:56:03.605744 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.605715 2580 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg"] Nov 04 01:56:03.607914 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.607886 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.660300 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.660262 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 01:56:03.660540 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.660519 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 01:56:03.660636 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.660522 2580 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vn697\"" Nov 04 01:56:03.660636 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.660559 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 01:56:03.709847 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.709812 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 01:56:03.711280 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.711255 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg"] Nov 04 01:56:03.711426 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.711410 2580 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 01:56:03.794196 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.794157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.794386 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.794228 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.794386 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.794287 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l9b2\" (UniqueName: \"kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.794386 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.794326 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.895454 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.895377 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.895454 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.895422 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.895663 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.895524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.895663 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.895561 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9b2\" (UniqueName: \"kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.896297 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.896181 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.898285 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.898263 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.898382 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.898361 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:03.959160 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:03.959132 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9b2\" (UniqueName: \"kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:04.221141 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:04.221064 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:04.605684 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:04.605646 2580 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg"] Nov 04 01:56:04.640291 ip-10-0-134-176 kubenswrapper[2580]: W1104 01:56:04.640257 2580 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7299f7d4_b1ef_4a5d_a8fd_883550532d2c.slice/crio-6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43 WatchSource:0}: Error finding container 6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43: Status 404 returned error can't find the container with id 6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43 Nov 04 01:56:05.124759 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:05.124714 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" event={"ID":"7299f7d4-b1ef-4a5d-a8fd-883550532d2c","Type":"ContainerStarted","Data":"6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43"} Nov 04 01:56:06.130197 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:06.130112 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkzgd" event={"ID":"636d2d6a-86f9-41ad-95f8-2b79b2645544","Type":"ContainerStarted","Data":"4e7b6b3c2d7b5ea5bd28cf0f0c108b5c191d7b7e29839def65f6542735e976f2"} Nov 04 01:56:06.130600 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:06.130344 2580 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:06.247285 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:06.246512 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nkzgd" podStartSLOduration=66.173195749 podStartE2EDuration="1m11.246492232s" totalImagesPullingTime="5.073296483s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:54:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:06.245854848 +0000 UTC m=+71.945953967" watchObservedRunningTime="2026-11-04 01:56:06.246492232 +0000 UTC m=+71.946591353" Nov 04 01:56:07.137232 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:07.137195 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" event={"ID":"7299f7d4-b1ef-4a5d-a8fd-883550532d2c","Type":"ContainerStarted","Data":"2569d1e37c54ca4983cedf04ed2680cb4b89d2ab234d542888cf30ee784282c3"} Nov 04 01:56:07.137688 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:07.137266 2580 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" event={"ID":"7299f7d4-b1ef-4a5d-a8fd-883550532d2c","Type":"ContainerStarted","Data":"3904708a82f90b398790f712f077dc0afebd2f896cbe73d0586542b7499170b8"} Nov 04 01:56:07.255095 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:07.255037 2580 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" podStartSLOduration=2.388055312 podStartE2EDuration="4.255018567s" totalImagesPullingTime="1.866963255s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:07.253634502 +0000 UTC m=+72.953733669" watchObservedRunningTime="2026-11-04 01:56:07.255018567 +0000 UTC m=+72.955117688" Nov 04 01:56:07.437420 ip-10-0-134-176 kubenswrapper[2580]: I1104 01:56:07.437290 2580 dynamic_cafile_content.go:175] "Shutting down controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 01:56:07.437454 ip-10-0-134-176 systemd[1]: Stopping Kubernetes Kubelet... Nov 04 01:56:07.455871 ip-10-0-134-176 systemd[1]: kubelet.service: Deactivated successfully. Nov 04 01:56:07.456051 ip-10-0-134-176 systemd[1]: Stopped Kubernetes Kubelet. Nov 04 01:56:07.456107 ip-10-0-134-176 systemd[1]: kubelet.service: Consumed 3.329s CPU time. Nov 04 01:56:07.472106 ip-10-0-134-176 systemd[1]: Starting Kubernetes Kubelet... Nov 04 01:56:07.537175 ip-10-0-134-176 kubenswrapper[5029]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:56:07.537175 ip-10-0-134-176 kubenswrapper[5029]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Nov 04 01:56:07.537175 ip-10-0-134-176 kubenswrapper[5029]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:56:07.537175 ip-10-0-134-176 kubenswrapper[5029]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Nov 04 01:56:07.539232 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539150 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.539232 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539232 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.539232 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539235 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539239 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539242 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539244 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539247 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539250 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539253 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539255 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539258 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539260 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539263 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539265 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539268 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539271 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539274 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539277 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539279 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539282 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539285 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539293 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.539326 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539296 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539298 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539301 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539304 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539306 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539309 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539311 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539314 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539317 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539319 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539322 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539326 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539331 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539334 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539338 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539341 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539343 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539346 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539348 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.539874 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539351 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539353 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539356 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539360 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539364 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539367 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539370 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539372 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539375 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539377 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539380 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539383 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539388 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539390 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539393 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539396 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539399 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539401 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539404 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.540432 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539406 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539409 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539412 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539415 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539418 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539420 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539423 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539425 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539428 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539430 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539433 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539436 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539438 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539441 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539443 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539446 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539448 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539451 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539453 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539455 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.540936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539458 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539460 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539463 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539515 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539521 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539524 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539526 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539529 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539532 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539534 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539537 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539540 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539542 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539545 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539547 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539551 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539553 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539556 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539558 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.541598 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539561 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539564 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539567 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539570 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539573 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539576 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.539584 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540379 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540386 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540389 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540391 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540394 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540397 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540400 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540402 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540406 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540410 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540413 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540416 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.542154 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540419 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540421 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540424 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540426 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540429 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540431 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540434 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540437 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540439 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540442 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540445 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540450 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540454 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540458 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540461 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540463 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540490 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540494 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540497 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.542662 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540501 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540508 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540511 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540514 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540517 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540519 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540522 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540525 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540527 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540530 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540533 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540535 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540538 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540540 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540543 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540545 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540548 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540550 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540553 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.543148 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540555 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540558 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540560 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540563 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540566 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540568 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540571 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540573 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540576 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540578 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540581 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540584 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540588 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540590 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540593 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540602 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540605 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540608 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540610 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540613 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.543621 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540615 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540618 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540621 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540623 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540626 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540628 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540631 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540633 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540636 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540638 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540641 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540643 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540646 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540648 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540650 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540653 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540656 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540658 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540661 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.544104 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540663 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540666 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540669 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540671 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540674 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540678 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540681 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540683 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540686 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540694 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540697 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540699 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540702 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540704 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540706 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540709 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.540712 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541255 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541263 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.544589 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541266 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541269 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541272 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541275 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541277 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541280 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541283 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541286 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541291 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541294 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541296 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541299 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541302 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541304 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541307 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541311 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541315 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541320 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541326 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541330 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.545063 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541333 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541335 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541339 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541341 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541344 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541346 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541349 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541351 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541354 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541356 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541359 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541361 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541363 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541366 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541369 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541371 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541375 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541379 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541382 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541385 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.545566 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541389 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541394 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541399 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541404 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541408 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541410 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541412 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541415 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541420 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541423 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541427 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541430 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541433 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541435 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541439 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541444 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541448 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541451 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541454 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.546062 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541456 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541459 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541461 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541481 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541485 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541488 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541491 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541493 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541496 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541498 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541501 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541503 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541506 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541510 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541512 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541515 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541518 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541520 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541523 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.546531 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541525 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541528 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541530 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541533 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541536 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541538 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541541 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541543 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541546 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541549 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541552 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541555 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541557 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541560 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541562 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541565 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541567 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541570 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541572 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.546998 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541574 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541577 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541579 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541582 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541584 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541587 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.541589 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541699 5029 flags.go:64] FLAG: --address="0.0.0.0" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541708 5029 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541715 5029 flags.go:64] FLAG: --anonymous-auth="true" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541720 5029 flags.go:64] FLAG: --application-metrics-count-limit="100" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541724 5029 flags.go:64] FLAG: --authentication-token-webhook="false" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541727 5029 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541731 5029 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541736 5029 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541739 5029 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541742 5029 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541745 5029 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541749 5029 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541752 5029 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541755 5029 flags.go:64] FLAG: --cgroup-root="" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541757 5029 flags.go:64] FLAG: --cgroups-per-qos="true" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541760 5029 flags.go:64] FLAG: --client-ca-file="" Nov 04 01:56:07.547516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541763 5029 flags.go:64] FLAG: --cloud-provider="external" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541767 5029 flags.go:64] FLAG: --cluster-dns="[]" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541771 5029 flags.go:64] FLAG: --cluster-domain="" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541774 5029 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541777 5029 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541780 5029 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541784 5029 flags.go:64] FLAG: --container-log-max-files="5" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541787 5029 flags.go:64] FLAG: --container-log-max-size="10Mi" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541790 5029 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541793 5029 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541796 5029 flags.go:64] FLAG: --containerd-namespace="k8s.io" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541799 5029 flags.go:64] FLAG: --contention-profiling="false" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541803 5029 flags.go:64] FLAG: --cpu-cfs-quota="true" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541805 5029 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541808 5029 flags.go:64] FLAG: --cpu-manager-policy="none" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541811 5029 flags.go:64] FLAG: --cpu-manager-policy-options="" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541817 5029 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541820 5029 flags.go:64] FLAG: --enable-controller-attach-detach="true" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541823 5029 flags.go:64] FLAG: --enable-debugging-handlers="true" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541826 5029 flags.go:64] FLAG: --enable-load-reader="false" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541829 5029 flags.go:64] FLAG: --enable-server="true" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541832 5029 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541857 5029 flags.go:64] FLAG: --event-burst="100" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541863 5029 flags.go:64] FLAG: --event-qps="50" Nov 04 01:56:07.548194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541866 5029 flags.go:64] FLAG: --event-storage-age-limit="default=0" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541869 5029 flags.go:64] FLAG: --event-storage-event-limit="default=0" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541872 5029 flags.go:64] FLAG: --eviction-hard="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541876 5029 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541880 5029 flags.go:64] FLAG: --eviction-minimum-reclaim="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541883 5029 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541886 5029 flags.go:64] FLAG: --eviction-soft="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541889 5029 flags.go:64] FLAG: --eviction-soft-grace-period="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541892 5029 flags.go:64] FLAG: --exit-on-lock-contention="false" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541895 5029 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541899 5029 flags.go:64] FLAG: --experimental-mounter-path="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541902 5029 flags.go:64] FLAG: --fail-cgroupv1="true" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541905 5029 flags.go:64] FLAG: --fail-swap-on="true" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541908 5029 flags.go:64] FLAG: --feature-gates="" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541912 5029 flags.go:64] FLAG: --file-check-frequency="20s" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541915 5029 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541918 5029 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541921 5029 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541924 5029 flags.go:64] FLAG: --healthz-port="10248" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541927 5029 flags.go:64] FLAG: --help="false" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541930 5029 flags.go:64] FLAG: --hostname-override="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541933 5029 flags.go:64] FLAG: --housekeeping-interval="10s" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541936 5029 flags.go:64] FLAG: --http-check-frequency="20s" Nov 04 01:56:07.548808 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541939 5029 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541944 5029 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541947 5029 flags.go:64] FLAG: --image-gc-high-threshold="85" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541950 5029 flags.go:64] FLAG: --image-gc-low-threshold="80" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541953 5029 flags.go:64] FLAG: --image-service-endpoint="" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541955 5029 flags.go:64] FLAG: --kernel-memcg-notification="false" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541958 5029 flags.go:64] FLAG: --kube-api-burst="100" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541961 5029 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541966 5029 flags.go:64] FLAG: --kube-api-qps="50" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541968 5029 flags.go:64] FLAG: --kube-reserved="" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541971 5029 flags.go:64] FLAG: --kube-reserved-cgroup="" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541974 5029 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541977 5029 flags.go:64] FLAG: --kubelet-cgroups="" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541980 5029 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541982 5029 flags.go:64] FLAG: --lock-file="" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541985 5029 flags.go:64] FLAG: --log-cadvisor-usage="false" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541988 5029 flags.go:64] FLAG: --log-flush-frequency="5s" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541991 5029 flags.go:64] FLAG: --log-json-info-buffer-size="0" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.541997 5029 flags.go:64] FLAG: --log-json-split-stream="false" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542000 5029 flags.go:64] FLAG: --log-text-info-buffer-size="0" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542003 5029 flags.go:64] FLAG: --log-text-split-stream="false" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542006 5029 flags.go:64] FLAG: --logging-format="text" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542009 5029 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542012 5029 flags.go:64] FLAG: --make-iptables-util-chains="true" Nov 04 01:56:07.549390 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542015 5029 flags.go:64] FLAG: --manifest-url="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542018 5029 flags.go:64] FLAG: --manifest-url-header="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542022 5029 flags.go:64] FLAG: --max-housekeeping-interval="15s" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542025 5029 flags.go:64] FLAG: --max-open-files="1000000" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542029 5029 flags.go:64] FLAG: --max-pods="110" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542031 5029 flags.go:64] FLAG: --maximum-dead-containers="-1" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542034 5029 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542037 5029 flags.go:64] FLAG: --memory-manager-policy="None" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542040 5029 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542044 5029 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542048 5029 flags.go:64] FLAG: --node-ip="0.0.0.0" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542050 5029 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542057 5029 flags.go:64] FLAG: --node-status-max-images="50" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542060 5029 flags.go:64] FLAG: --node-status-update-frequency="10s" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542063 5029 flags.go:64] FLAG: --oom-score-adj="-999" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542066 5029 flags.go:64] FLAG: --pod-cidr="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542070 5029 flags.go:64] FLAG: --pod-manifest-path="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542073 5029 flags.go:64] FLAG: --pod-max-pids="-1" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542076 5029 flags.go:64] FLAG: --pods-per-core="0" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542079 5029 flags.go:64] FLAG: --port="10250" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542082 5029 flags.go:64] FLAG: --protect-kernel-defaults="false" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542084 5029 flags.go:64] FLAG: --provider-id="aws:///us-west-1a/i-0e66ea304c654fcb5" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542087 5029 flags.go:64] FLAG: --qos-reserved="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542090 5029 flags.go:64] FLAG: --read-only-port="10255" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542093 5029 flags.go:64] FLAG: --register-node="true" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542096 5029 flags.go:64] FLAG: --register-with-taints="" Nov 04 01:56:07.550129 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542100 5029 flags.go:64] FLAG: --registry-burst="10" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542103 5029 flags.go:64] FLAG: --registry-qps="5" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542106 5029 flags.go:64] FLAG: --reserved-cpus="" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542109 5029 flags.go:64] FLAG: --reserved-memory="" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542113 5029 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542116 5029 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542119 5029 flags.go:64] FLAG: --rotate-certificates="false" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542122 5029 flags.go:64] FLAG: --rotate-server-certificates="false" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542124 5029 flags.go:64] FLAG: --runonce="false" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542127 5029 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542130 5029 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542133 5029 flags.go:64] FLAG: --seccomp-default="false" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542136 5029 flags.go:64] FLAG: --serialize-image-pulls="true" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542139 5029 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542141 5029 flags.go:64] FLAG: --storage-driver-db="cadvisor" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542145 5029 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542148 5029 flags.go:64] FLAG: --storage-driver-password="root" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542151 5029 flags.go:64] FLAG: --storage-driver-secure="false" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542154 5029 flags.go:64] FLAG: --storage-driver-table="stats" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542157 5029 flags.go:64] FLAG: --storage-driver-user="root" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542160 5029 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542163 5029 flags.go:64] FLAG: --sync-frequency="1m0s" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542167 5029 flags.go:64] FLAG: --system-cgroups="" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542170 5029 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542175 5029 flags.go:64] FLAG: --system-reserved-cgroup="" Nov 04 01:56:07.550780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542178 5029 flags.go:64] FLAG: --tls-cert-file="" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542181 5029 flags.go:64] FLAG: --tls-cipher-suites="[]" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542185 5029 flags.go:64] FLAG: --tls-min-version="" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542188 5029 flags.go:64] FLAG: --tls-private-key-file="" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542191 5029 flags.go:64] FLAG: --topology-manager-policy="none" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542194 5029 flags.go:64] FLAG: --topology-manager-policy-options="" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542196 5029 flags.go:64] FLAG: --topology-manager-scope="container" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542199 5029 flags.go:64] FLAG: --v="2" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542204 5029 flags.go:64] FLAG: --version="false" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542208 5029 flags.go:64] FLAG: --vmodule="" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542212 5029 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542215 5029 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542345 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542349 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542352 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542355 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542357 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542360 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542362 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542365 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542367 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542370 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.551379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542374 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542377 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542379 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542381 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542384 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542387 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542389 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542394 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542396 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542399 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542402 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542404 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542407 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542409 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542412 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542415 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542417 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542419 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542422 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542424 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.551936 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542427 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542430 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542432 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542435 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542438 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542440 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542444 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542447 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542450 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542453 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542456 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542458 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542462 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542481 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542485 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542488 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542491 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542494 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542497 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.552557 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542501 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542503 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542506 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542508 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542511 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542514 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542516 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542519 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542521 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542524 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542526 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542529 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542531 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542534 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542536 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542538 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542543 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542547 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542550 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.553113 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542553 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542556 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542558 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542560 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542563 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542565 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542569 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542571 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542574 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542576 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542579 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542582 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542584 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542588 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542590 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542593 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542595 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542597 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542599 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542602 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.553832 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542604 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542607 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542609 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542612 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542614 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542617 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542619 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542621 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542624 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542626 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542629 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542632 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542634 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542637 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542639 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542642 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542644 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.554343 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542647 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.542675 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542710 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542714 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542717 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542719 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542722 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542724 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542728 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542731 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542733 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542736 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542738 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542741 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542743 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542746 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.554826 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542748 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542751 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542753 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542756 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542758 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542760 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542763 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542765 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542768 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542770 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542773 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542776 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542778 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542781 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542783 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542785 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542788 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542790 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542794 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542796 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.555379 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542799 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542801 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542803 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542806 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542809 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542812 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542814 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542817 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542819 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542821 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542824 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542827 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542829 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542833 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542837 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542840 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542842 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542844 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542847 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.555992 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542849 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542852 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542854 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542857 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542860 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542863 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542866 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542868 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542871 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542873 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542876 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542878 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542881 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542883 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542886 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542889 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542891 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542893 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542896 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542898 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.556537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542900 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542903 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542906 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542909 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542913 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542915 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542918 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542921 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542923 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542925 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542928 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542931 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542933 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542935 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542938 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542940 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542943 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542946 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542948 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.557068 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542951 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542953 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542956 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542958 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542960 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542963 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542965 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542968 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542970 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542973 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542976 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542978 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542980 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.542983 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.546736 5029 server.go:525] "Kubelet version" kubeletVersion="v1.35.9" Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.546751 5029 server.go:527] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.546769 5029 watchdog_linux.go:95] "Systemd watchdog is not enabled" Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.546774 5029 watchdog_linux.go:138] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546848 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546854 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.557537 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546857 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546861 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546864 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546867 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546870 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546873 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546876 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546879 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546881 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546884 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546887 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546891 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546894 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546896 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546899 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546902 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546904 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546907 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546911 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.558071 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546915 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546918 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546920 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546923 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546926 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546928 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546931 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546934 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546939 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546943 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546946 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546948 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546951 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546953 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546956 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546958 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546961 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546963 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546966 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.558538 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546968 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546971 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546974 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546976 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546979 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546982 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546985 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546988 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546990 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546993 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546995 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.546998 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547000 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547003 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547006 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547008 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547011 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547014 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547016 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.558981 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547019 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547022 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547024 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547027 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547029 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547031 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547034 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547036 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547039 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547041 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547043 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547046 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547048 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547051 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547054 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547056 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547059 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547061 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547063 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547066 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.559434 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547069 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547071 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547073 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547076 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547078 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547080 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547083 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547085 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547088 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547090 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547093 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547095 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547098 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547100 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547103 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547105 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547108 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547110 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547112 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547115 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.559935 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547117 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547119 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547122 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547124 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547127 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547129 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547132 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.547158 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547966 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547985 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547990 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.547996 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548000 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548004 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548008 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548012 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.560571 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548017 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548021 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548031 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548036 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548039 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548043 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548049 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548053 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548057 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548062 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548066 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548070 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548075 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548079 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548083 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548095 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548102 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548108 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548113 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.561127 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548118 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548123 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548127 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548132 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548136 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548138 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548142 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548145 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548151 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548153 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548156 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548159 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548161 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548164 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548167 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548169 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548172 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548174 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548177 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548180 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.561870 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548182 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548187 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548190 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548193 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548196 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548198 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548201 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548204 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548210 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548213 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548216 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548219 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548221 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548224 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548229 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548232 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548234 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548237 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548239 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548243 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.562603 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548245 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548248 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548251 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548255 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548257 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548260 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548265 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548268 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548270 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548273 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548276 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548278 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548281 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548284 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548287 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548289 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548292 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548294 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548297 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548302 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.563337 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548306 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548310 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548314 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548317 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548320 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548323 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548325 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548328 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548330 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548333 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548335 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548341 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548344 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548346 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548349 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548352 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548354 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548357 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.563831 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548360 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.548405 5029 feature_gate.go:465] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false DRAPartitionableDevices:false EventedPLEG:false KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:true MutatingAdmissionPolicy:false RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true UserNamespacesSupport:true]} Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548503 5029 feature_gate.go:386] unrecognized feature gate: NewOLMBoxCutterRuntime Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548511 5029 feature_gate.go:386] unrecognized feature gate: ImageStreamImportMode Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548514 5029 feature_gate.go:386] unrecognized feature gate: NoOverlayMode Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548517 5029 feature_gate.go:386] unrecognized feature gate: SignatureStores Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548520 5029 feature_gate.go:386] unrecognized feature gate: AutomatedEtcdBackup Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548523 5029 feature_gate.go:386] unrecognized feature gate: AdditionalStorageConfig Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548526 5029 feature_gate.go:386] unrecognized feature gate: InsightsConfig Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548529 5029 feature_gate.go:386] unrecognized feature gate: MultiArchInstallAzure Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548532 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryptionProvider Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548534 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNSInstall Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548537 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementVSphere Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548539 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUpstreamParity Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548542 5029 feature_gate.go:386] unrecognized feature gate: NewOLMConfigAPI Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548546 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementGCP Nov 04 01:56:07.564268 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548552 5029 feature_gate.go:386] unrecognized feature gate: ClusterVersionOperatorConfiguration Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548554 5029 feature_gate.go:386] unrecognized feature gate: AzureClusterHostedDNSInstall Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548557 5029 feature_gate.go:386] unrecognized feature gate: VolumeGroupSnapshot Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548559 5029 feature_gate.go:386] unrecognized feature gate: OpenShiftPodSecurityAdmission Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548562 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstallIBMCloud Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548565 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerification Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548567 5029 feature_gate.go:386] unrecognized feature gate: MetricsCollectionProfiles Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548570 5029 feature_gate.go:386] unrecognized feature gate: NetworkConnect Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548573 5029 feature_gate.go:386] unrecognized feature gate: NoRegistryClusterInstall Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548575 5029 feature_gate.go:386] unrecognized feature gate: MultiDiskSetup Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548578 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiNetworks Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548581 5029 feature_gate.go:386] unrecognized feature gate: ShortCertRotation Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548586 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiDisk Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548589 5029 feature_gate.go:386] unrecognized feature gate: EtcdBackendQuota Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548592 5029 feature_gate.go:386] unrecognized feature gate: NewOLMPreflightPermissionChecks Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548594 5029 feature_gate.go:386] unrecognized feature gate: BootImageSkewEnforcement Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548597 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIControlPlaneInstall Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548599 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpointsInstall Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548602 5029 feature_gate.go:386] unrecognized feature gate: NutanixMultiSubnets Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548605 5029 feature_gate.go:386] unrecognized feature gate: DualReplica Nov 04 01:56:07.564922 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548607 5029 feature_gate.go:386] unrecognized feature gate: AzureDedicatedHosts Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548610 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIComputeInstall Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548612 5029 feature_gate.go:386] unrecognized feature gate: IngressControllerDynamicConfigurationManager Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548615 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIInstall Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548617 5029 feature_gate.go:386] unrecognized feature gate: ExternalSnapshotMetadata Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548622 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAWS Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548625 5029 feature_gate.go:386] unrecognized feature gate: EVPN Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548628 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdatePreflight Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548631 5029 feature_gate.go:386] unrecognized feature gate: EventTTL Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548634 5029 feature_gate.go:386] unrecognized feature gate: InsightsOnDemandDataGather Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548636 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCExternalClaimsSourcing Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548640 5029 feature_gate.go:386] unrecognized feature gate: ConfidentialCluster Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548642 5029 feature_gate.go:386] unrecognized feature gate: BootcNodeManagement Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548645 5029 feature_gate.go:386] unrecognized feature gate: ConfigurablePKI Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548648 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationVSphere Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548650 5029 feature_gate.go:386] unrecognized feature gate: TLSAdherence Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548653 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigration Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548655 5029 feature_gate.go:386] unrecognized feature gate: ProvisioningRequestAvailable Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548660 5029 feature_gate.go:386] unrecognized feature gate: AWSDedicatedHosts Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548663 5029 feature_gate.go:386] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Nov 04 01:56:07.565722 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548666 5029 feature_gate.go:386] unrecognized feature gate: AWSClusterHostedDNS Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548669 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDC Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548671 5029 feature_gate.go:386] unrecognized feature gate: OVNObservability Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548674 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementPowerVS Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548677 5029 feature_gate.go:386] unrecognized feature gate: ImageModeStatusReporting Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548680 5029 feature_gate.go:386] unrecognized feature gate: ClusterMonitoringConfig Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548683 5029 feature_gate.go:407] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548687 5029 feature_gate.go:386] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548692 5029 feature_gate.go:386] unrecognized feature gate: NewOLM Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548697 5029 feature_gate.go:386] unrecognized feature gate: UpgradeStatus Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548700 5029 feature_gate.go:386] unrecognized feature gate: VSphereMixedNodeEnv Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548703 5029 feature_gate.go:386] unrecognized feature gate: KMSEncryption Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548705 5029 feature_gate.go:386] unrecognized feature gate: SigstoreImageVerificationPKI Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548708 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementAzure Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548710 5029 feature_gate.go:386] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548713 5029 feature_gate.go:386] unrecognized feature gate: AzureDualStackInstall Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548715 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementBareMetal Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548718 5029 feature_gate.go:386] unrecognized feature gate: OSStreams Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548721 5029 feature_gate.go:386] unrecognized feature gate: AWSDualStackInstall Nov 04 01:56:07.566300 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548723 5029 feature_gate.go:386] unrecognized feature gate: UserNamespacesPodSecurityStandards Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548726 5029 feature_gate.go:386] unrecognized feature gate: GatewayAPIWithoutOLM Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548730 5029 feature_gate.go:386] unrecognized feature gate: Example2 Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548735 5029 feature_gate.go:386] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548740 5029 feature_gate.go:386] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548743 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagement Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548746 5029 feature_gate.go:386] unrecognized feature gate: AzureMultiDisk Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548749 5029 feature_gate.go:386] unrecognized feature gate: CRDCompatibilityRequirementOperator Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548751 5029 feature_gate.go:386] unrecognized feature gate: DNSNameResolver Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548753 5029 feature_gate.go:386] unrecognized feature gate: ManagedBootImagesCPMS Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548756 5029 feature_gate.go:386] unrecognized feature gate: NewOLMOwnSingleNamespace Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548759 5029 feature_gate.go:386] unrecognized feature gate: GCPDualStackInstall Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548761 5029 feature_gate.go:386] unrecognized feature gate: ConsolePluginContentSecurityPolicy Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548764 5029 feature_gate.go:386] unrecognized feature gate: MixedCPUsAllocation Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548766 5029 feature_gate.go:386] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548769 5029 feature_gate.go:386] unrecognized feature gate: GCPCustomAPIEndpoints Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548774 5029 feature_gate.go:386] unrecognized feature gate: OnPremDNSRecords Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548777 5029 feature_gate.go:386] unrecognized feature gate: ClusterUpdateAcceptRisks Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548780 5029 feature_gate.go:386] unrecognized feature gate: VSphereHostVMGroupZonal Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548783 5029 feature_gate.go:386] unrecognized feature gate: Example Nov 04 01:56:07.566843 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548785 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationAWS Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548788 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548791 5029 feature_gate.go:386] unrecognized feature gate: MachineAPIMigrationOpenStack Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548793 5029 feature_gate.go:386] unrecognized feature gate: AWSEuropeanSovereignCloudInstall Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548796 5029 feature_gate.go:386] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548798 5029 feature_gate.go:386] unrecognized feature gate: ClusterAPIMachineManagementOpenStack Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548801 5029 feature_gate.go:386] unrecognized feature gate: VSphereMultiVCenterDay2 Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548804 5029 feature_gate.go:386] unrecognized feature gate: AzureWorkloadIdentity Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548811 5029 feature_gate.go:409] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548815 5029 feature_gate.go:386] unrecognized feature gate: CRIOCredentialProviderConfig Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548818 5029 feature_gate.go:386] unrecognized feature gate: BuildCSIVolumes Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548821 5029 feature_gate.go:386] unrecognized feature gate: IrreconcilableMachineConfig Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:07.548824 5029 feature_gate.go:386] unrecognized feature gate: GCPSovereignCloudInstall Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.548889 5029 server.go:957] "Client rotation is on, will bootstrap in background" Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550368 5029 bootstrap.go:85] "Current kubeconfig file contents are still valid, no bootstrap necessary" Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550427 5029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-client-current.pem" Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550628 5029 server.go:1014] "Starting client certificate rotation" Nov 04 01:56:07.567502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550760 5029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550815 5029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-11-03 01:49:55 +0000 UTC" deadline="2028-07-16 02:07:12.102056244 +0000 UTC" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.550848 5029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14880h11m4.551211992s" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.551051 5029 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.552075 5029 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.553274 5029 log.go:25] "Validated CRI v1 runtime API" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.555067 5029 log.go:25] "Validated CRI v1 image API" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.555805 5029 server.go:1424] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Nov 04 01:56:07.568223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.557505 5029 fs.go:135] Filesystem UUIDs: map[23976f5f-675a-40b9-a5f9-a29f45c91abb:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 e2c22dab-699c-4abd-813e-e99e726c1b55:/dev/nvme0n1p4] Nov 04 01:56:07.568785 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.557523 5029 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:24 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:26 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77/userdata/shm major:0 minor:281 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/099ab16ecfe60ce11d78c63be8796037b3e4f3052981cf895edf7cd696027225/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/099ab16ecfe60ce11d78c63be8796037b3e4f3052981cf895edf7cd696027225/userdata/shm major:0 minor:208 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e/userdata/shm major:0 minor:51 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1/userdata/shm major:0 minor:225 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e/userdata/shm major:0 minor:92 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6/userdata/shm major:0 minor:72 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d/userdata/shm major:0 minor:96 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5/userdata/shm major:0 minor:78 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa/userdata/shm major:0 minor:77 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43/userdata/shm major:0 minor:305 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f/userdata/shm major:0 minor:266 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f/userdata/shm major:0 minor:279 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222/userdata/shm major:0 minor:52 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d/userdata/shm major:0 minor:62 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c/userdata/shm major:0 minor:76 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e/userdata/shm major:0 minor:230 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96/userdata/shm major:0 minor:283 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6/userdata/shm major:0 minor:202 fsType:tmpfs blockSize:0} /run/containers/storage/overlay-containers/fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1/userdata/shm:{mountpoint:/run/containers/storage/overlay-containers/fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1/userdata/shm major:0 minor:88 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:32 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/etc-tuned:{mountpoint:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/etc-tuned major:0 minor:48 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/tmp:{mountpoint:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/tmp major:0 minor:47 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~projected/kube-api-access-bs62d:{mountpoint:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~projected/kube-api-access-bs62d major:0 minor:59 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volume-subpaths/run-systemd/ovnkube-controller/6:{mountpoint:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volume-subpaths/run-systemd/ovnkube-controller/6 major:0 minor:26 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~projected/kube-api-access-wqb6x:{mountpoint:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~projected/kube-api-access-wqb6x major:0 minor:57 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~secret/ovn-node-metrics-cert:{mountpoint:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~secret/ovn-node-metrics-cert major:0 minor:49 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~projected/kube-api-access-kdrmc:{mountpoint:/var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~projected/kube-api-access-kdrmc major:0 minor:201 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~secret/metrics-tls:{mountpoint:/var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~secret/metrics-tls major:0 minor:220 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/22ce2354-ad7a-4100-970f-ce84512cce41/volumes/kubernetes.io~projected/kube-api-access-hgnnc:{mountpoint:/var/lib/kubelet/pods/22ce2354-ad7a-4100-970f-ce84512cce41/volumes/kubernetes.io~projected/kube-api-access-hgnnc major:0 minor:53 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/38e16a3b-bcac-4482-b177-eb0061845a74/volumes/kubernetes.io~secret/original-pull-secret:{mountpoint:/var/lib/kubelet/pods/38e16a3b-bcac-4482-b177-eb0061845a74/volumes/kubernetes.io~secret/original-pull-secret major:0 minor:272 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/447db221-9de8-4e44-b228-ab76255a229d/volumes/kubernetes.io~secret/agent-certs:{mountpoint:/var/lib/kubelet/pods/447db221-9de8-4e44-b228-ab76255a229d/volumes/kubernetes.io~secret/agent-certs major:0 minor:50 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/636d2d6a-86f9-41ad-95f8-2b79b2645544/volumes/kubernetes.io~projected/kube-api-access-7xdjn:{mountpoint:/var/lib/kubelet/pods/636d2d6a-86f9-41ad-95f8-2b79b2645544/volumes/kubernetes.io~projected/kube-api-access-7xdjn major:0 minor:278 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~projected/kube-api-access-4l9b2:{mountpoint:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~projected/kube-api-access-4l9b2 major:0 minor:304 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config:{mountpoint:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config major:0 minor:303 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-tls:{mountpoint:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-tls major:0 minor:299 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7a827277-7c60-4dbd-b5a5-fc861a2c236d/volumes/kubernetes.io~projected/kube-api-access-jksbb:{mountpoint:/var/lib/kubelet/pods/7a827277-7c60-4dbd-b5a5-fc861a2c236d/volumes/kubernetes.io~projected/kube-api-access-jksbb major:0 minor:61 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f/volumes/kubernetes.io~projected/kube-api-access-8647w:{mountpoint:/var/lib/kubelet/pods/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f/volumes/kubernetes.io~projected/kube-api-access-8647w major:0 minor:207 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/bound-sa-token:{mountpoint:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/bound-sa-token major:0 minor:264 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/kube-api-access-4tmtz:{mountpoint:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/kube-api-access-4tmtz major:0 minor:265 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/registry-tls:{mountpoint:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/registry-tls major:0 minor:262 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/image-registry-private-configuration:{mountpoint:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/image-registry-private-configuration major:0 minor:258 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/installation-pull-secrets:{mountpoint:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/installation-pull-secrets major:0 minor:263 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a3e06791-0981-4894-9ac1-d55aec5adf00/volumes/kubernetes.io~projected/kube-api-access-h2vnh:{mountpoint:/var/lib/kubelet/pods/a3e06791-0981-4894-9ac1-d55aec5adf00/volumes/kubernetes.io~projected/kube-api-access-h2vnh major:0 minor:55 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~projected/kube-api-access-fwbcd:{mountpoint:/var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~projected/kube-api-access-fwbcd major:0 minor:58 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~secret/metrics-certs:{mountpoint:/var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~secret/metrics-certs major:0 minor:277 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~projected/kube-api-access-jbqdz:{mountpoint:/var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~projected/kube-api-access-jbqdz major:0 minor:206 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~secret/insights-runtime-extractor-tls:{mountpoint:/var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~secret/insights-runtime-extractor-tls major:0 minor:229 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ece54c3b-fb1f-4f03-9430-22f897a99c91/volumes/kubernetes.io~projected/kube-api-access-s8vbd:{mountpoint:/var/lib/kubelet/pods/ece54c3b-fb1f-4f03-9430-22f897a99c91/volumes/kubernetes.io~projected/kube-api-access-s8vbd major:0 minor:54 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60/volumes/kubernetes.io~projected/kube-api-access-skv9p:{mountpoint:/var/lib/kubelet/pods/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60/volumes/kubernetes.io~projected/kube-api-access-skv9p major:0 minor:60 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~projected/kube-api-access-6qgt4:{mountpoint:/var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~projected/kube-api-access-6qgt4 major:0 minor:197 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~secret/cert:{mountpoint:/var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~secret/cert major:0 minor:45 fsType:tmpfs blockSize:0} /var/lib/kubelet/pods/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0/volumes/kubernetes.io~projected/kube-api-access-2vxdx:{mountpoint:/var/lib/kubelet/pods/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0/volumes/kubernetes.io~projected/kube-api-access-2vxdx major:0 minor:56 fsType:tmpfs blockSize:0} composefs_0-34:{mountpoint:/ major:0 minor:34 fsType:overlay blockSize:0} overlay_0-100:{mountpoint:/var/lib/containers/storage/overlay/396252c1e0b337d5e0e50f6a0e49de312c68de06d51c4aa9f3cc49ad21cd9c84/merged major:0 minor:100 fsType:overlay blockSize:0} overlay_0-102:{mountpoint:/var/lib/containers/storage/overlay/d012b5a4b42ceb35d7ac71f746426eb1e533ce106f9726761e327264de5ccac9/merged major:0 minor:102 fsType:overlay blockSize:0} overlay_0-104:{mountpoint:/var/lib/containers/storage/overlay/d3b914d26cf2167fb4d3a7794892dcd5bd2168af6c7cf8c7c4844a15daa6b86f/merged major:0 minor:104 fsType:overlay blockSize:0} overlay_0-109:{mountpoint:/var/lib/containers/storage/overlay/8c07ca73e9d84345985e0d7f3157a42145fa1434b00c9fbf9532626c5969ccab/merged major:0 minor:109 fsType:overlay blockSize:0} overlay_0-113:{mountpoint:/var/lib/containers/storage/overlay/cdb732682498e702d78021155f017203611fafe965ac390ab05abe9cb67e092e/merged major:0 minor:113 fsType:overlay blockSize:0} overlay_0-119:{mountpoint:/var/lib/containers/storage/overlay/af660f69c04f869006645c406f8e9a83fb9eec588c211c07ccbd733d54944dc9/merged major:0 minor:119 fsType:overlay blockSize:0} overlay_0-124:{mountpoint:/var/lib/containers/storage/overlay/18a7ebb7ecd26ef624b16ff95c4bcc5a788312a3111b841bfb1fc396a34a67a9/merged major:0 minor:124 fsType:overlay blockSize:0} overlay_0-127:{mountpoint:/var/lib/containers/storage/overlay/474af8d2103b9837097493995721afa2546f9100f5dc3be14a31ac52ff834bf1/merged major:0 minor:127 fsType:overlay blockSize:0} overlay_0-132:{mountpoint:/var/lib/containers/storage/overlay/fb3bd19d480c1220890742d401d5556bdb7c2094194fc49eba50bf4e5788ebe0/merged major:0 minor:132 fsType:overlay blockSize:0} overlay_0-134:{mountpoint:/var/lib/containers/storage/overlay/e6138efc37e726f223de45777ac2cd3c82aa442ddaaf166bf02837a4ada08e4e/merged major:0 minor:134 fsType:overlay blockSize:0} overlay_0-144:{mountpoint:/var/lib/containers/storage/overlay/04ce39c628c551b5fcb2c495fd35918b0a19db64c252197c16d33d5bf20f3c25/merged major:0 minor:144 fsType:overlay blockSize:0} overlay_0-147:{mountpoint:/var/lib/containers/storage/overlay/ca27b5a406b75a2b2dba31147966c406561701089f8843d5c56cad9024562f58/merged major:0 minor:147 fsType:overlay blockSize:0} overlay_0-152:{mountpoint:/var/lib/containers/storage/overlay/0a915d654ddb4ed88c5a85877e1d5e72f4c884fdfb6ca81b9459d9c385a3fd18/merged major:0 minor:152 fsType:overlay blockSize:0} overlay_0-157:{mountpoint:/var/lib/containers/storage/overlay/21fd47429973ed89520481a2dd9e82eb0c60ccc643eb6daa3c7bfff72b980aa9/merged major:0 minor:157 fsType:overlay blockSize:0} overlay_0-158:{mountpoint:/var/lib/containers/storage/overlay/46540d7b7e2bd4174708fe8276a8e27c86ec55660aaa66305dd4c681c0933224/merged major:0 minor:158 fsType:overlay blockSize:0} overlay_0-167:{mountpoint:/var/lib/containers/storage/overlay/0f9d378a466ef32aebabbdaf6c8395278fa0be0f42587ae5e99c6a6d611bfe75/merged major:0 minor:167 fsType:overlay blockSize:0} overlay_0-174:{mountpoint:/var/lib/containers/storage/overlay/c2339d8b7de5d233e84f2d98a262201d497dcafd25a02433e7df7177d1b9b4d5/merged major:0 minor:174 fsType:overlay blockSize:0} overlay_0-179:{mountpoint:/var/lib/containers/storage/overlay/ad97bbb22ee6cdf219a69257ab17f54f90236deb36f8b0a131add02bac20716b/merged major:0 minor:179 fsType:overlay blockSize:0} overlay_0-182:{mountpoint:/var/lib/containers/storage/overlay/e9c629b154f4abb41a21183a3f55739958106222be99b4a52fe05cf9c98f5160/merged major:0 minor:182 fsType:overlay blockSize:0} overlay_0-187:{mountpoint:/var/lib/containers/storage/overlay/6b740947d42e957d9f45d998e91f8283f10459c752cf655907aae0c209e329f5/merged major:0 minor:187 fsType:overlay blockSize:0} overlay_0-192:{mountpoint:/var/lib/containers/storage/overlay/5e0dba1fafc7801fe724851a2512523f2a44aff4cb1e1ec05de2c06b8faf4a14/merged major:0 minor:192 fsType:overlay blockSize:0} overlay_0-204:{mountpoint:/var/lib/containers/storage/overlay/516d4bc6b1321b7722ca59a2c6f75eec6f57a737c2065d91cdc9dc504f76280f/merged major:0 minor:204 fsType:overlay blockSize:0} overlay_0-210:{mountpoint:/var/lib/containers/storage/overlay/ffa6c956e417da3867a7ecc4e60b0e65179d73f7525ccd3071c84cb3e5925165/merged major:0 minor:210 fsType:overlay blockSize:0} overlay_0-212:{mountpoint:/var/lib/containers/storage/overlay/62e0048d6b2851767ba43161522c3772f0b1397d21c6bdd103831a7af38e6e4d/merged major:0 minor:212 fsType:overlay blockSize:0} overlay_0-214:{mountpoint:/var/lib/containers/storage/overlay/71a90e014ff1e2759f0b59be86b2c950e7bd4956f8af39f8a05d49e318904538/merged major:0 minor:214 fsType:overlay blockSize:0} overlay_0-227:{mountpoint:/var/lib/containers/storage/overlay/ffb8f213d56ed8e5075aa0d8c2e6adf04ae0f90af1794b746583413a965e53c9/merged major:0 minor:227 fsType:overlay blockSize:0} overlay_0-232:{mountpoint:/var/lib/containers/storage/overlay/8559bd7cdf47981cf7d8f084010948c5817d5f2257bc8b4b85dabc1d7f33f2a0/merged major:0 minor:232 fsType:overlay blockSize:0} overlay_0-234:{mountpoint:/var/lib/containers/storage/overlay/195f8587b0d9c117b8460b0aad4c5e4392edef9eb90b6f05e910dc66543926b6/merged major:0 minor:234 fsType:overlay blockSize:0} overlay_0-236:{mountpoint:/var/lib/containers/storage/overlay/0bd2a8440eceecb9946a3c7ea085dd4054da65fa5b4f1c61dd82498d56ed97bb/merged major:0 minor:236 fsType:overlay blockSize:0} overlay_0-242:{mountpoint:/var/lib/containers/storage/overlay/d98e830939969857bb0ddb30d510fba7f74a3c30f7b95b47ab3a49ed79e8ba61/merged major:0 minor:242 fsType:overlay blockSize:0} overlay_0-248:{mountpoint:/var/lib/containers/storage/overlay/9668739f02c85dfd9ce2c14a971b49bdab0a07c61aa21c695304c9825238257f/merged major:0 minor:248 fsType:overlay blockSize:0} overlay_0-253:{mountpoint:/var/lib/containers/storage/overlay/0038dd3541225df34a649ca9a6d50cb0c9f7f40e8a10ef1e375bd08b3f40c8b9/merged major:0 minor:253 fsType:overlay blockSize:0} overlay_0-268:{mountpoint:/var/lib/containers/storage/overlay/f34a102f83a3c2cd8c2c76182cf888dd528e4dd526ba450540dcded133430d6d/merged major:0 minor:268 fsType:overlay blockSize:0} overlay_0-270:{mountpoint:/var/lib/containers/storage/overlay/07f05f658139ba62c5a917ff58702698227e90bf2b45246e321a692955ebc001/merged major:0 minor:270 fsType:overlay blockSize:0} overlay_0-285:{mountpoint:/var/lib/containers/storage/overlay/67719740bb46c8b0dbcb24902c881f646114b75f7d0852b62b8ef32149f3387f/merged major:0 minor:285 fsType:overlay blockSize:0} overlay_0-287:{mountpoint:/var/lib/containers/storage/overlay/86eb8249759c38e929c94796906e8fc549af3825c8beea2fb18dc21d1eec2e84/merged major:0 minor:287 fsType:overlay blockSize:0} overlay_0-289:{mountpoint:/var/lib/containers/storage/overlay/cbf23e54155dea911ceb4417ef3596b5a3653c33793f7e20488d2a2da5cc2165/merged major:0 minor:289 fsType:overlay blockSize:0} overlay_0-291:{mountpoint:/var/lib/containers/storage/overlay/8775e04b153641eeab56490ba6588e4612bdbcdcfca3ad6ee3b5f3ae958b6687/merged major:0 minor:291 fsType:overlay blockSize:0} overlay_0-293:{mountpoint:/var/lib/containers/storage/overlay/fe12598f25a1934ca3409ac2318825c79d4a1d0d8f357bd40d35b2cfa42402ee/merged major:0 minor:293 fsType:overlay blockSize:0} overlay_0-307:{mountpoint:/var/lib/containers/storage/overlay/8dcbca3698953cd381c0e441e78d09e183be8a1018b41a7a3519f9f9360b2d3c/merged major:0 minor:307 fsType:overlay blockSize:0} overlay_0-309:{mountpoint:/var/lib/containers/storage/overlay/cd5d3f05eedc9704eec19e180b7a0d8545d4fc4a608676c3c9237a020b67d2dc/merged major:0 minor:309 fsType:overlay blockSize:0} overlay_0-311:{mountpoint:/var/lib/containers/storage/overlay/a684a954c39ca3c844f4475a05d4cc08213356775c6bd0b070f1a8689fcafc61/merged major:0 minor:311 fsType:overlay blockSize:0} overlay_0-317:{mountpoint:/var/lib/containers/storage/overlay/3843c5f05dbd85ff23a4efb57ecaada2f6f94a0ceb785a45557ee0682918c4e1/merged major:0 minor:317 fsType:overlay blockSize:0} overlay_0-323:{mountpoint:/var/lib/containers/storage/overlay/828edaf63fcdf6512eef3e6d9b51404600179b9a85da2420c0e1ce01ae080278/merged major:0 minor:323 fsType:overlay blockSize:0} overlay_0-66:{mountpoint:/var/lib/containers/storage/overlay/b710d23bc63e5b0aed59fb7e77aee25badc295d66be7a73d4267dce411aa1967/merged major:0 minor:66 fsType:overlay blockSize:0} overlay_0-68:{mountpoint:/var/lib/containers/storage/overlay/f8eeaa7346a223d4d00af17e4ea414259e34a1a43a7e81bc33f5874cd6257fce/merged major:0 minor:68 fsType:overlay blockSize:0} overlay_0-70:{mountpoint:/var/lib/containers/storage/overlay/ce9fa472875e3dd2b84c6d05116a10de81d67009059e151f80d6c2f65e78849b/merged major:0 minor:70 fsType:overlay blockSize:0} overlay_0-74:{mountpoint:/var/lib/containers/storage/overlay/160d6e3d11bb1295d0b4799a556845bfbe6f4f2c9a8de7b372e1e4f96df2ed50/merged major:0 minor:74 fsType:overlay blockSize:0} overlay_0-82:{mountpoint:/var/lib/containers/storage/overlay/426cfb681b333883b05163c02893cb6fb0db0ea307a540b30d7fdcfd20b7ce24/merged major:0 minor:82 fsType:overlay blockSize:0} overlay_0-84:{mountpoint:/var/lib/containers/storage/overlay/bc14cda6fb5f44d9b542f95bae1dafa0dfc6af4a203d17a567e9978380d1ddd3/merged major:0 minor:84 fsType:overlay blockSize:0} overlay_0-86:{mountpoint:/var/lib/containers/storage/overlay/5c685a836d0b149d83a26707b1b3a65e3c32ce2ff06f0dbbb0644775b25183c8/merged major:0 minor:86 fsType:overlay blockSize:0} overlay_0-90:{mountpoint:/var/lib/containers/storage/overlay/aaf62d5f10dbbb2a3d319125562e1fa4b531a90a17c5e2d69a7ed406d4e4b884/merged major:0 minor:90 fsType:overlay blockSize:0} overlay_0-94:{mountpoint:/var/lib/containers/storage/overlay/4dadbad80ed8576179fb43baca3158a25cde11087da8292a0edf0f08274194ad/merged major:0 minor:94 fsType:overlay blockSize:0} overlay_0-98:{mountpoint:/var/lib/containers/storage/overlay/c33bc6906c70dd5edbbb3282de4f2756b0d0bcff48890026874660f00bdaa5c4/merged major:0 minor:98 fsType:overlay blockSize:0}] Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.563736 5029 manager.go:217] Machine: {Timestamp:2026-11-04 01:56:07.563272467 +0000 UTC m=+0.056597599 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3125171 MemoryCapacity:32812019712 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec28e9e6a901cc9ebb20825136bf1dcb SystemUUID:ec28e9e6-a901-cc9e-bb20-825136bf1dcb BootID:f5eae5a2-4c9e-41d4-bc80-00af8495082f Filesystems:[{Device:overlay_0-182 DeviceMajor:0 DeviceMinor:182 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ee29ba49f4fb7c151258d73fb67b2d360041ae4c2e17c5126f90fdcbb7b8d3f6/userdata/shm DeviceMajor:0 DeviceMinor:202 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~projected/kube-api-access-jbqdz DeviceMajor:0 DeviceMinor:206 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-270 DeviceMajor:0 DeviceMinor:270 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/636d2d6a-86f9-41ad-95f8-2b79b2645544/volumes/kubernetes.io~projected/kube-api-access-7xdjn DeviceMajor:0 DeviceMinor:278 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/tmp DeviceMajor:0 DeviceMinor:47 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-70 DeviceMajor:0 DeviceMinor:70 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-157 DeviceMajor:0 DeviceMinor:157 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-174 DeviceMajor:0 DeviceMinor:174 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2ce9575184c28aa4a697149d3809e79f139bcdba1a9e085de84a8dfc2ca6f2a1/userdata/shm DeviceMajor:0 DeviceMinor:225 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-227 DeviceMajor:0 DeviceMinor:227 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~secret/metrics-certs DeviceMajor:0 DeviceMinor:277 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-285 DeviceMajor:0 DeviceMinor:285 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-34 DeviceMajor:0 DeviceMinor:34 Capacity:6422528 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/run/containers/storage/overlay-containers/cd89eeacd5b1edb4e03e7deb02a27029e55bb071f679ba3d1143c4eeb79f604d/userdata/shm DeviceMajor:0 DeviceMinor:62 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-109 DeviceMajor:0 DeviceMinor:109 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-119 DeviceMajor:0 DeviceMinor:119 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/099ab16ecfe60ce11d78c63be8796037b3e4f3052981cf895edf7cd696027225/userdata/shm DeviceMajor:0 DeviceMinor:208 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-248 DeviceMajor:0 DeviceMinor:248 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-287 DeviceMajor:0 DeviceMinor:287 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-289 DeviceMajor:0 DeviceMinor:289 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/22ce2354-ad7a-4100-970f-ce84512cce41/volumes/kubernetes.io~projected/kube-api-access-hgnnc DeviceMajor:0 DeviceMinor:53 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/7a827277-7c60-4dbd-b5a5-fc861a2c236d/volumes/kubernetes.io~projected/kube-api-access-jksbb DeviceMajor:0 DeviceMinor:61 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-84 DeviceMajor:0 DeviceMinor:84 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-98 DeviceMajor:0 DeviceMinor:98 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-187 DeviceMajor:0 DeviceMinor:187 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-210 DeviceMajor:0 DeviceMinor:210 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-214 DeviceMajor:0 DeviceMinor:214 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-253 DeviceMajor:0 DeviceMinor:253 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0/volumes/kubernetes.io~projected/kube-api-access-2vxdx DeviceMajor:0 DeviceMinor:56 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/5ffdb2f6c134f71341eb0105f03606ce17e218ea914f72f8b8c93ca96ce8f38d/userdata/shm DeviceMajor:0 DeviceMinor:96 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-132 DeviceMajor:0 DeviceMinor:132 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-158 DeviceMajor:0 DeviceMinor:158 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~secret/cert DeviceMajor:0 DeviceMinor:45 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/e376fe9328cb0194564dbd7473fff994e05aefa85018c228da530f057b7da40e/userdata/shm DeviceMajor:0 DeviceMinor:230 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-242 DeviceMajor:0 DeviceMinor:242 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-307 DeviceMajor:0 DeviceMinor:307 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/a3e06791-0981-4894-9ac1-d55aec5adf00/volumes/kubernetes.io~projected/kube-api-access-h2vnh DeviceMajor:0 DeviceMinor:55 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-179 DeviceMajor:0 DeviceMinor:179 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~projected/kube-api-access-4l9b2 DeviceMajor:0 DeviceMinor:304 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-309 DeviceMajor:0 DeviceMinor:309 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:24 Capacity:16406007808 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~empty-dir/etc-tuned DeviceMajor:0 DeviceMinor:48 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~secret/ovn-node-metrics-cert DeviceMajor:0 DeviceMinor:49 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60/volumes/kubernetes.io~projected/kube-api-access-skv9p DeviceMajor:0 DeviceMinor:60 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/68ccff4693b93d665f20af98d7261d8a05cffab27adf0264f789cd01dcb65bfa/userdata/shm DeviceMajor:0 DeviceMinor:77 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-90 DeviceMajor:0 DeviceMinor:90 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/9b7fb45c6036112bf382cbf61cebac7e66f06a26238d9188256c260db5eea26f/userdata/shm DeviceMajor:0 DeviceMinor:279 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-291 DeviceMajor:0 DeviceMinor:291 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/078d2895-52e5-4a17-943a-ab3de7a3e40e/volumes/kubernetes.io~projected/kube-api-access-bs62d DeviceMajor:0 DeviceMinor:59 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e/userdata/shm DeviceMajor:0 DeviceMinor:51 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-66 DeviceMajor:0 DeviceMinor:66 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-100 DeviceMajor:0 DeviceMinor:100 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-144 DeviceMajor:0 DeviceMinor:144 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-152 DeviceMajor:0 DeviceMinor:152 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/00154cfa0297c19d28738fa3f225f02d9c65dd0d10f1f6c95870d18df5023b77/userdata/shm DeviceMajor:0 DeviceMinor:281 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volumes/kubernetes.io~projected/kube-api-access-wqb6x DeviceMajor:0 DeviceMinor:57 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-68 DeviceMajor:0 DeviceMinor:68 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5/userdata/shm DeviceMajor:0 DeviceMinor:78 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/2e6c9c04f6fcbb038c33edfa1ca5b8c93518d69d9703afc9bcab49fc4ca96e5e/userdata/shm DeviceMajor:0 DeviceMinor:92 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-192 DeviceMajor:0 DeviceMinor:192 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/6a9b54a1c5e9a958d741201e7cb0b9d55429e69954425d9b8f3a567deaed3e43/userdata/shm DeviceMajor:0 DeviceMinor:305 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/de209566e7cded1d7ed171d25b824ac7c4c16e823635f8da69249a6db7939d8c/userdata/shm DeviceMajor:0 DeviceMinor:76 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/fc1ed243222e2dff4ec626e6ae2ad951471e9c4c207197c991487fe9efa6ded1/userdata/shm DeviceMajor:0 DeviceMinor:88 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-167 DeviceMajor:0 DeviceMinor:167 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-kube-rbac-proxy-config DeviceMajor:0 DeviceMinor:303 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-311 DeviceMajor:0 DeviceMinor:311 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-82 DeviceMajor:0 DeviceMinor:82 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-268 DeviceMajor:0 DeviceMinor:268 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-293 DeviceMajor:0 DeviceMinor:293 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-323 DeviceMajor:0 DeviceMinor:323 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/447db221-9de8-4e44-b228-ab76255a229d/volumes/kubernetes.io~secret/agent-certs DeviceMajor:0 DeviceMinor:50 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-74 DeviceMajor:0 DeviceMinor:74 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-102 DeviceMajor:0 DeviceMinor:102 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-104 DeviceMajor:0 DeviceMinor:104 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-127 DeviceMajor:0 DeviceMinor:127 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-204 DeviceMajor:0 DeviceMinor:204 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f/volumes/kubernetes.io~projected/kube-api-access-8647w DeviceMajor:0 DeviceMinor:207 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-212 DeviceMajor:0 DeviceMinor:212 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:overlay_0-94 DeviceMajor:0 DeviceMinor:94 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-236 DeviceMajor:0 DeviceMinor:236 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/image-registry-private-configuration DeviceMajor:0 DeviceMinor:258 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/registry-tls DeviceMajor:0 DeviceMinor:262 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/bound-sa-token DeviceMajor:0 DeviceMinor:264 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~projected/kube-api-access-4tmtz DeviceMajor:0 DeviceMinor:265 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/38e16a3b-bcac-4482-b177-eb0061845a74/volumes/kubernetes.io~secret/original-pull-secret DeviceMajor:0 DeviceMinor:272 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/var/lib/kubelet/pods/ece54c3b-fb1f-4f03-9430-22f897a99c91/volumes/kubernetes.io~projected/kube-api-access-s8vbd DeviceMajor:0 DeviceMinor:54 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-124 DeviceMajor:0 DeviceMinor:124 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:overlay_0-147 DeviceMajor:0 DeviceMinor:147 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~projected/kube-api-access-kdrmc DeviceMajor:0 DeviceMinor:201 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/840c4e7c-424c-4f46-8f30-cb39f2d31a24/volumes/kubernetes.io~secret/installation-pull-secrets DeviceMajor:0 DeviceMinor:263 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/94201dc63bd9485cebf1278e1dd02ef7e9a4fe8850e659eb5ea7865d5f38176f/userdata/shm DeviceMajor:0 DeviceMinor:266 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ece5118afbce818695566d4fa50e513fc295fb0468dc42bd6b902f64f069bf96/userdata/shm DeviceMajor:0 DeviceMinor:283 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-113 DeviceMajor:0 DeviceMinor:113 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/bb2c7ac1-0806-4d8c-8c19-979d3c080f80/volumes/kubernetes.io~secret/insights-runtime-extractor-tls DeviceMajor:0 DeviceMinor:229 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-232 DeviceMajor:0 DeviceMinor:232 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/7299f7d4-b1ef-4a5d-a8fd-883550532d2c/volumes/kubernetes.io~secret/prometheus-operator-tls DeviceMajor:0 DeviceMinor:299 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-317 DeviceMajor:0 DeviceMinor:317 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/run/containers/storage/overlay-containers/ba4f177f365095a7bed6bfd9852e7631217db1ff8f7d29a6d8cc9d9f01b53222/userdata/shm DeviceMajor:0 DeviceMinor:52 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:/run/containers/storage/overlay-containers/390eb8f2bc3f1ee72d1fb0e17ade08eb83c9eedd39aaf2a2cfe837f44d4e71a6/userdata/shm DeviceMajor:0 DeviceMinor:72 Capacity:67108864 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-86 DeviceMajor:0 DeviceMinor:86 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/var/lib/kubelet/pods/0829f7ce-7f76-4727-bb67-7f60f08a3189/volume-subpaths/run-systemd/ovnkube-controller/6 DeviceMajor:0 DeviceMinor:26 Capacity:6562406400 Type:vfs Inodes:819200 HasInodes:true} {Device:/var/lib/kubelet/pods/f456ccdc-f27b-4026-b425-891f5c46263e/volumes/kubernetes.io~projected/kube-api-access-6qgt4 DeviceMajor:0 DeviceMinor:197 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:/var/lib/kubelet/pods/0a1d99ff-e629-46da-a1e0-e92fd128a237/volumes/kubernetes.io~secret/metrics-tls DeviceMajor:0 DeviceMinor:220 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-234 DeviceMajor:0 DeviceMinor:234 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:32 Capacity:16406011904 Type:vfs Inodes:1048576 HasInodes:true} {Device:/var/lib/kubelet/pods/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17/volumes/kubernetes.io~projected/kube-api-access-fwbcd DeviceMajor:0 DeviceMinor:58 Capacity:31633420288 Type:vfs Inodes:4005373 HasInodes:true} {Device:overlay_0-134 DeviceMajor:0 DeviceMinor:134 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:00154cfa0297c19 MacAddress:d6:ad:4f:93:57:ba Speed:10000 Mtu:8901} {Name:099ab16ecfe60ce MacAddress:de:f3:dc:70:67:3b Speed:10000 Mtu:8901} {Name:2ce9575184c28aa MacAddress:d2:7b:78:0f:f0:24 Speed:10000 Mtu:8901} {Name:6a9b54a1c5e9a95 MacAddress:7a:9b:ef:57:20:d3 Speed:10000 Mtu:8901} {Name:94201dc63bd9485 MacAddress:0a:6e:dc:c1:20:2a Speed:10000 Mtu:8901} {Name:9b7fb45c6036112 MacAddress:ae:95:79:81:e4:25 Speed:10000 Mtu:8901} {Name:br-ex MacAddress:02:78:27:26:be:a5 Speed:0 Mtu:9001} {Name:br-int MacAddress:22:92:5d:33:0d:a7 Speed:0 Mtu:8901} {Name:e376fe9328cb019 MacAddress:ae:d9:5e:22:8f:36 Speed:10000 Mtu:8901} {Name:ece5118afbce818 MacAddress:46:24:06:05:70:4f Speed:10000 Mtu:8901} {Name:ee29ba49f4fb7c1 MacAddress:16:c0:c9:a9:a5:3f Speed:10000 Mtu:8901} {Name:ens5 MacAddress:02:78:27:26:be:a5 Speed:0 Mtu:9001} {Name:genev_sys_6081 MacAddress:7a:4a:d5:6f:0c:b7 Speed:0 Mtu:65000} {Name:ovn-k8s-mp0 MacAddress:0a:58:0a:85:00:02 Speed:0 Mtu:8901} {Name:ovs-system MacAddress:52:c7:b4:e5:50:ca Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812019712 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564026 5029 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564065 5029 manager.go:233] Version: {KernelVersion:5.14.0-687.60.1.el9_8.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.8.20261023-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564574 5029 container_manager_linux.go:275] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564640 5029 container_manager_linux.go:280] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-176.us-west-1.compute.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"/system.slice","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{},"system-reserved-compressible":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564948 5029 topology_manager.go:143] "Creating topology manager with none policy" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564961 5029 container_manager_linux.go:311] "Creating device plugin manager" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.564975 5029 manager.go:144] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.565001 5029 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.565100 5029 container_manager_linux.go:320] "Creating Dynamic Resource Allocation (DRA) manager" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.565339 5029 state_mem.go:41] "Initialized" logger="CPUManager state memory" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.565923 5029 server.go:1235] "Using root directory" path="/var/lib/kubelet" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566133 5029 kubelet.go:493] "Attempting to sync node with API server" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566149 5029 kubelet.go:394] "Adding static pod path" path="/etc/kubernetes/manifests" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566160 5029 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566170 5029 kubelet.go:405] "Adding apiserver pod source" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566178 5029 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566831 5029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.566841 5029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 01:56:07.569093 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.567334 5029 kuberuntime_manager.go:304] "Container runtime initialized" containerRuntime="cri-o" version="1.35.10-3.rhaos4.22.git9383502.el9" apiVersion="v1" Nov 04 01:56:07.570416 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570399 5029 state_checkpoint.go:81] "State checkpoint: restored pod resource state from checkpoint" Nov 04 01:56:07.570416 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570414 5029 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Nov 04 01:56:07.570564 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570548 5029 certificate_store.go:147] "Loading cert/key pair from a file" filePath="/var/lib/kubelet/pki/kubelet-server-current.pem" Nov 04 01:56:07.570774 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570758 5029 kubelet.go:963] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Nov 04 01:56:07.570834 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570797 5029 kubelet.go:990] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Nov 04 01:56:07.570925 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570913 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570933 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570944 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570953 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570962 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570971 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570981 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.570990 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571002 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571012 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Nov 04 01:56:07.571034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571032 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Nov 04 01:56:07.571449 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571046 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Nov 04 01:56:07.571449 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571081 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Nov 04 01:56:07.571449 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.571090 5029 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Nov 04 01:56:07.574561 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.574545 5029 server.go:1263] "Started kubelet" Nov 04 01:56:07.574646 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.574611 5029 server.go:182] "Starting to listen" address="0.0.0.0" port=10250 Nov 04 01:56:07.574749 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.574702 5029 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Nov 04 01:56:07.574796 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.574768 5029 server_v1.go:49] "podresources" method="list" useActivePods=true Nov 04 01:56:07.575081 ip-10-0-134-176 systemd[1]: Started Kubernetes Kubelet. Nov 04 01:56:07.575227 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.575200 5029 server.go:254] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Nov 04 01:56:07.575814 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.575694 5029 server.go:317] "Adding debug handlers to kubelet server" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.578601 5029 fs_resource_analyzer.go:69] "Starting FS ResourceAnalyzer" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.578779 5029 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.579027 5029 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-11-03 01:49:56 +0000 UTC" deadline="2028-05-10 08:24:35.895378413 +0000 UTC" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.579066 5029 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13278h28m28.316316133s" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.579119 5029 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.579241 5029 volume_manager.go:309] "The desired_state_of_world populator starts" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.579249 5029 volume_manager.go:311] "Starting Kubelet Volume Manager" Nov 04 01:56:07.580069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.579363 5029 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.581954 5029 factory.go:153] Registering CRI-O factory Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582048 5029 factory.go:223] Registration of the crio container factory successfully Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582128 5029 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582139 5029 factory.go:55] Registering systemd factory Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582161 5029 factory.go:223] Registration of the systemd container factory successfully Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582183 5029 factory.go:103] Registering Raw factory Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582205 5029 manager.go:1199] Started watching for new ooms in manager Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582687 5029 manager.go:319] Starting recovery of all containers Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582712 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22ce2354-ad7a-4100-970f-ce84512cce41" volumeName="kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582761 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7299f7d4-b1ef-4a5d-a8fd-883550532d2c" volumeName="kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582777 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb2c7ac1-0806-4d8c-8c19-979d3c080f80" volumeName="kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582820 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ece54c3b-fb1f-4f03-9430-22f897a99c91" volumeName="kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582880 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7299f7d4-b1ef-4a5d-a8fd-883550532d2c" volumeName="kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582911 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7a827277-7c60-4dbd-b5a5-fc861a2c236d" volumeName="kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582932 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582944 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582962 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb2c7ac1-0806-4d8c-8c19-979d3c080f80" volumeName="kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582982 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1d99ff-e629-46da-a1e0-e92fd128a237" volumeName="kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.582998 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" volumeName="kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs" seLinuxMountContext="" Nov 04 01:56:07.583036 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.583009 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f456ccdc-f27b-4026-b425-891f5c46263e" volumeName="kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4" seLinuxMountContext="" Nov 04 01:56:07.584657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584631 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1d99ff-e629-46da-a1e0-e92fd128a237" volumeName="kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584664 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22ce2354-ad7a-4100-970f-ce84512cce41" volumeName="kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584676 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="38e16a3b-bcac-4482-b177-eb0061845a74" volumeName="kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584687 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7299f7d4-b1ef-4a5d-a8fd-883550532d2c" volumeName="kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584698 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584706 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584714 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584721 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3e06791-0981-4894-9ac1-d55aec5adf00" volumeName="kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584741 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22ce2354-ad7a-4100-970f-ce84512cce41" volumeName="kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc" seLinuxMountContext="" Nov 04 01:56:07.584745 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584749 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="447db221-9de8-4e44-b228-ab76255a229d" volumeName="kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584757 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7299f7d4-b1ef-4a5d-a8fd-883550532d2c" volumeName="kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584764 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584772 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb2c7ac1-0806-4d8c-8c19-979d3c080f80" volumeName="kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584779 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="bb2c7ac1-0806-4d8c-8c19-979d3c080f80" volumeName="kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584789 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f456ccdc-f27b-4026-b425-891f5c46263e" volumeName="kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584811 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0" volumeName="kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584825 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="22ce2354-ad7a-4100-970f-ce84512cce41" volumeName="kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584842 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078d2895-52e5-4a17-943a-ab3de7a3e40e" volumeName="kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584851 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078d2895-52e5-4a17-943a-ab3de7a3e40e" volumeName="kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584859 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0829f7ce-7f76-4727-bb67-7f60f08a3189" volumeName="kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584867 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0" volumeName="kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584879 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0829f7ce-7f76-4727-bb67-7f60f08a3189" volumeName="kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584888 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="447db221-9de8-4e44-b228-ab76255a229d" volumeName="kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584895 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="7cc1dde7-35d0-4bec-8ead-b06ecff9f71f" volumeName="kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584906 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584914 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="840c4e7c-424c-4f46-8f30-cb39f2d31a24" volumeName="kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584921 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a3e06791-0981-4894-9ac1-d55aec5adf00" volumeName="kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584931 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60" volumeName="kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584944 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60" volumeName="kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584957 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="078d2895-52e5-4a17-943a-ab3de7a3e40e" volumeName="kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584970 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0829f7ce-7f76-4727-bb67-7f60f08a3189" volumeName="kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584984 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1d99ff-e629-46da-a1e0-e92fd128a237" volumeName="kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.584998 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="636d2d6a-86f9-41ad-95f8-2b79b2645544" volumeName="kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585009 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="a9dfb0df-5635-4cbd-81e6-6c55f20b3a17" volumeName="kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585020 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ece54c3b-fb1f-4f03-9430-22f897a99c91" volumeName="kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585054 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="ece54c3b-fb1f-4f03-9430-22f897a99c91" volumeName="kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585066 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0829f7ce-7f76-4727-bb67-7f60f08a3189" volumeName="kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585080 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0829f7ce-7f76-4727-bb67-7f60f08a3189" volumeName="kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585091 5029 reconstruct.go:129] "Volume is marked as uncertain and added into the actual state" pod="" podName="0a1d99ff-e629-46da-a1e0-e92fd128a237" volumeName="kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc" seLinuxMountContext="" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585103 5029 reconstruct.go:97] "Volume reconstruction finished" Nov 04 01:56:07.585163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.585110 5029 reconciler.go:29] "Reconciler: start to sync state" Nov 04 01:56:07.611324 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.611277 5029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Nov 04 01:56:07.612968 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.612942 5029 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Nov 04 01:56:07.612968 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.612964 5029 status_manager.go:249] "Starting to sync pod status with apiserver" Nov 04 01:56:07.613142 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.612984 5029 kubelet.go:2521] "Starting kubelet main sync loop" Nov 04 01:56:07.613142 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.613024 5029 kubelet.go:2545] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Nov 04 01:56:07.627433 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.627402 5029 generic.go:356] "Generic (PLEG): container finished" podID="7db4f44b8ad055392b513e6afd9c7331" containerID="e2d379f9629553af3b15b4a46f8869e263ace0e89639392d81b77847e22c26ea" exitCode=0 Nov 04 01:56:07.657923 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657892 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="9da1ceb6b63df8a94678da5bf44f124873835f435ba89b100b3e9878289d9729" exitCode=0 Nov 04 01:56:07.657923 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657919 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="af8d03a5719bfa9d86f95214a9ff558fc701206b063a0203547ca331e09b0489" exitCode=0 Nov 04 01:56:07.657923 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657928 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="51a18529b960f26bf6f43505ece722f45d84b1ac4133ba05d3c7f7840a485c74" exitCode=0 Nov 04 01:56:07.658145 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657936 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="00fad87fe5bbdcb0e08d0eba319989838a3dd094ebdabec80cb33b932a7762db" exitCode=0 Nov 04 01:56:07.658145 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657945 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="d8f5ad2f6616a3141ec7ed4292613fe2c3cb6630a02758dc0513dceab4901f41" exitCode=0 Nov 04 01:56:07.658145 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.657951 5029 generic.go:356] "Generic (PLEG): container finished" podID="22ce2354-ad7a-4100-970f-ce84512cce41" containerID="44f591229c5e114b9f62e2e99195cf370f34d25675613c447b87f4f0abb09138" exitCode=0 Nov 04 01:56:07.665657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.665640 5029 manager.go:324] Recovery completed Nov 04 01:56:07.679682 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.679654 5029 kubelet_node_status.go:442] "Error getting the current node from lister" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:56:07.681781 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.681766 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.686219 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.686201 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.686306 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.686235 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.686306 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.686248 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.688510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688446 5029 cpu_manager.go:226] "Starting" policy="none" Nov 04 01:56:07.688510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688459 5029 cpu_manager.go:227] "Reconciling" reconcilePeriod="10s" Nov 04 01:56:07.688510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688493 5029 state_mem.go:41] "Initialized" logger="CPUManager state checkpoint.CPUManager state memory" Nov 04 01:56:07.688632 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688460 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.688632 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688609 5029 state_mem.go:94] "Updated default CPUSet" logger="CPUManager state checkpoint.CPUManager state memory" cpuSet="" Nov 04 01:56:07.688632 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688621 5029 state_mem.go:102] "Updated CPUSet assignments" logger="CPUManager state checkpoint.CPUManager state memory" assignments={} Nov 04 01:56:07.688727 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688639 5029 state_checkpoint.go:142] "restored state from checkpoint" logger="CPUManager state checkpoint" Nov 04 01:56:07.688727 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688647 5029 state_checkpoint.go:143] "defaultCPUSet" logger="CPUManager state checkpoint" defaultCpuSet="" Nov 04 01:56:07.688727 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688655 5029 policy_none.go:50] "Start" Nov 04 01:56:07.688727 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688664 5029 memory_manager.go:187] "Starting memorymanager" policy="None" Nov 04 01:56:07.688727 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688673 5029 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Nov 04 01:56:07.688945 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688798 5029 state_mem.go:77] "Updated machine memory state" logger="Memory Manager state checkpoint" Nov 04 01:56:07.688945 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688808 5029 state_checkpoint.go:85] "State checkpoint: restored state from checkpoint" logger="Memory Manager state checkpoint" Nov 04 01:56:07.688945 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.688817 5029 policy_none.go:44] "Start" Nov 04 01:56:07.689432 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.689416 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.689535 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.689449 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.689535 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.689463 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.693123 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.693100 5029 node_container_manager_linux.go:130] "Enforcing system reserved compressible on cgroup" cgroupName="/system.slice" limits={"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"} Nov 04 01:56:07.696898 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.696885 5029 manager.go:341] "Starting Device Plugin manager" Nov 04 01:56:07.696971 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.696908 5029 manager.go:525] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Nov 04 01:56:07.696971 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.696919 5029 server.go:87] "Starting device plugin registration server" Nov 04 01:56:07.697204 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697191 5029 eviction_manager.go:194] "Eviction manager: starting control loop" Nov 04 01:56:07.697253 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697203 5029 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Nov 04 01:56:07.697542 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697517 5029 plugin_watcher.go:53] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Nov 04 01:56:07.697642 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697617 5029 plugin_watcher.go:201] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Nov 04 01:56:07.697691 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697651 5029 plugin_manager.go:119] "The desired_state_of_world populator (plugin watcher) starts" Nov 04 01:56:07.697691 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697658 5029 plugin_manager.go:121] "Starting Kubelet Plugin Manager" Nov 04 01:56:07.697776 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.697700 5029 reconciler.go:167] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-11-04T01:56:07.697639257Z","UUID":"d4c37e21-ce3f-4f7a-a9eb-869e6d677fb2","Handler":null,"Name":"","Endpoint":""} Nov 04 01:56:07.698996 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.698978 5029 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Nov 04 01:56:07.699127 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.699004 5029 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Nov 04 01:56:07.699127 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.699044 5029 eviction_manager.go:272] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Nov 04 01:56:07.699127 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.699078 5029 eviction_manager.go:297] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" Nov 04 01:56:07.714584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.714118 5029 kubelet.go:2608] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal"] Nov 04 01:56:07.714679 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.714666 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.715429 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.715415 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.715522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.715442 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.715522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.715457 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.715747 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.715736 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.715894 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.715881 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.716460 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716437 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.716551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716437 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.716551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716490 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.716551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716502 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.716551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716506 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.716551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716520 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.716713 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.716668 5029 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.716752 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716717 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerStarted","Data":"f971cac9e265263b800f353ab662dc729b391a58d319ca3826137039fbb26827"} Nov 04 01:56:07.716752 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716745 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerDied","Data":"e2d379f9629553af3b15b4a46f8869e263ace0e89639392d81b77847e22c26ea"} Nov 04 01:56:07.716821 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716764 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"7db4f44b8ad055392b513e6afd9c7331","Type":"ContainerStarted","Data":"6568b23d473ddcd8d5b402e13323929596597a0a2c6b801f29893c2357deb0b5"} Nov 04 01:56:07.716821 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716796 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"3c019a8a489976b5a638075ae287a9c6","Type":"ContainerStarted","Data":"0211f3fe3fa0b4810003ece67154aefba6dfaf87ae8f295d9487adfc0beac5a6"} Nov 04 01:56:07.716821 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716808 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" event={"ID":"3c019a8a489976b5a638075ae287a9c6","Type":"ContainerStarted","Data":"1769c918035438d99823fb9cf9be1fa48b5922413dc887fc2c9a890213a9349e"} Nov 04 01:56:07.716911 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.716820 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.717515 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.717502 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.717579 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.717525 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.717579 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.717535 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.717671 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.717657 5029 kubelet.go:3364] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-176.us-west-1.compute.internal\" not found" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.776968 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.776942 5029 reflector.go:446] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:56:07.776968 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.776958 5029 reflector.go:446] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:56:07.777363 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.777348 5029 reconstruct.go:205] "DevicePaths of reconstructed volumes updated" Nov 04 01:56:07.777453 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.777438 5029 reflector.go:446] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:56:07.780589 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.780576 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.797748 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.797726 5029 kubelet_node_status.go:388] "Setting node annotation to enable volume controller attach/detach" Nov 04 01:56:07.798764 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.798746 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientMemory" Nov 04 01:56:07.798887 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.798781 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasNoDiskPressure" Nov 04 01:56:07.798887 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.798795 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeHasSufficientPID" Nov 04 01:56:07.798887 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.798850 5029 kubelet_node_status.go:77] "Attempting to register node" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.875495 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.875445 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:56:07.875643 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.875533 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.875643 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.875552 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.875779 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.875760 5029 reflector.go:446] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:161" Nov 04 01:56:07.877540 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.877517 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.877668 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.877558 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.877668 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.877583 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.927393 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.927365 5029 kubelet_node_status.go:126] "Node was previously registered" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.927520 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.927429 5029 kubelet_node_status.go:80] "Successfully registered node" node="ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.928043 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.928028 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeNotReady" Nov 04 01:56:07.928155 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.928058 5029 setters.go:546] "Node became not ready" node="ip-10-0-134-176.us-west-1.compute.internal" condition={"type":"Ready","status":"False","lastHeartbeatTime":"2026-11-04T01:56:07Z","lastTransitionTime":"2026-11-04T01:56:07Z","reason":"KubeletNotReady","message":"CSINode is not yet initialized"} Nov 04 01:56:07.941114 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.941065 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:56:07.941114 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:07.941110 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978354 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978330 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978354 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978282 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978495 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978396 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978495 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978418 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978495 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978458 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/3c019a8a489976b5a638075ae287a9c6-config\") pod \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"3c019a8a489976b5a638075ae287a9c6\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.978579 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.978504 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7db4f44b8ad055392b513e6afd9c7331-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" (UID: \"7db4f44b8ad055392b513e6afd9c7331\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:07.992838 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:07.992814 5029 nodeinfomanager.go:471] existing CSINode "ip-10-0-134-176.us-west-1.compute.internal" is owned by different node (oldNodeID="8783d135-b294-4c7c-88df-860fa2dc097b", newNodeID=""), cleaning up... Nov 04 01:56:08.567344 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.567314 5029 apiserver.go:52] "Watching apiserver" Nov 04 01:56:08.662571 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.662541 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:08.662723 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.662671 5029 kubelet.go:3368] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:08.670522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.670501 5029 reflector.go:446] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Nov 04 01:56:08.670987 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.670968 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-sd84m","openshift-dns/dns-default-p9lv2","openshift-insights/insights-runtime-extractor-mrgm4","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal","openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg","openshift-multus/multus-additional-cni-plugins-59clt","openshift-network-operator/iptables-alerter-zd9fs","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw","openshift-dns/node-resolver-vdgnv","openshift-cluster-node-tuning-operator/tuned-7s8zx","openshift-image-registry/image-registry-75fb56bf5f-nt7k7","kube-system/global-pull-secret-syncer-w2257","openshift-image-registry/node-ca-mbhp4","openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj","openshift-network-diagnostics/network-check-target-nkzgd","openshift-ovn-kubernetes/ovnkube-node-jm8wv","kube-system/konnectivity-agent-75hc6","kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal","openshift-multus/multus-zxf47","openshift-ingress-canary/ingress-canary-rjstr"] Nov 04 01:56:08.681883 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.681860 5029 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Nov 04 01:56:08.682011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.681907 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kdrmc\" (UniqueName: \"kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.682011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.681934 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.682011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.681951 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.682163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682038 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/840c4e7c-424c-4f46-8f30-cb39f2d31a24-ca-trust-extracted\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.682163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682069 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.682163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682110 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.682163 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682150 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2vxdx\" (UniqueName: \"kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682173 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682189 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682195 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-tmp-dir\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682206 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682230 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682254 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.682322 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682277 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682327 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682372 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682400 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682446 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682517 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682544 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682562 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.682584 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682578 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682594 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682609 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fwbcd\" (UniqueName: \"kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682640 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682659 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682675 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682687 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-data-volume\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682690 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682721 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682771 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682818 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682841 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hgnnc\" (UniqueName: \"kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682856 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682878 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8647w\" (UniqueName: \"kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w\") pod \"migrator-66df5d496b-zdttj\" (UID: \"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:56:08.682910 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682909 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682967 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682975 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/0a1d99ff-e629-46da-a1e0-e92fd128a237-tmp-dir\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.682992 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683012 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683026 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683059 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683104 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683155 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683196 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683222 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jksbb\" (UniqueName: \"kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683277 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4tmtz\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683387 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683432 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.683452 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683451 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683479 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s8vbd\" (UniqueName: \"kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683530 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6qgt4\" (UniqueName: \"kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683561 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683598 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683622 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683692 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683718 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683742 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683766 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683803 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jbqdz\" (UniqueName: \"kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683853 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.683956 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684009 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqb6x\" (UniqueName: \"kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684035 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.684079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684061 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684146 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684177 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684235 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684251 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684265 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684282 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684299 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l9b2\" (UniqueName: \"kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684341 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h2vnh\" (UniqueName: \"kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684397 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684420 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684436 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684451 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684489 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684537 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684577 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.684621 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684604 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684638 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684662 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684705 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684739 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684776 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684815 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684836 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684892 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-tmp\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684925 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-skv9p\" (UniqueName: \"kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684949 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-tuned\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.684973 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685078 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685115 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685155 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685175 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685191 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.685223 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685221 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685248 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685272 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685291 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685318 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685348 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685375 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685422 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685456 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685527 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bs62d\" (UniqueName: \"kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685544 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685560 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685574 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.685860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.685589 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.721589 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.721565 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:56:08.721688 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:08.721609 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal\" already exists" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:08.722089 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.722074 5029 warnings.go:107] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Nov 04 01:56:08.722142 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:08.722121 5029 kubelet.go:3370] "Failed creating a mirror pod" err="pods \"kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal\" already exists" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-176.us-west-1.compute.internal" Nov 04 01:56:08.726551 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.726522 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Nov 04 01:56:08.726665 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.726563 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.726665 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.726637 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.726665 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.726626 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.728020 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728004 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.728088 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728075 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.728522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728500 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728646 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728521 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.728646 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728527 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Nov 04 01:56:08.728646 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728579 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728592 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728700 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728710 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728714 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728717 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728754 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728768 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728778 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728798 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Nov 04 01:56:08.728830 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728817 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.729160 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728835 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Nov 04 01:56:08.729160 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728836 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.729160 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728847 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Nov 04 01:56:08.729160 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728855 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.729160 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.728889 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.733937 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.733919 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-binary-copy\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.734035 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.733982 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a3e06791-0981-4894-9ac1-d55aec5adf00-iptables-alerter-script\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.734082 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734067 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-env-overrides\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.734218 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734197 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-script-lib\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.734343 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734331 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-cni-binary-copy\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.734691 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734627 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-serviceca\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.734839 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734820 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-daemon-config\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.734989 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.734963 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.735076 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.735059 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovnkube-config\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.736497 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.736457 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/22ce2354-ad7a-4100-970f-ce84512cce41-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.736574 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.736499 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-certificates\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.769028 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769005 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Nov 04 01:56:08.769235 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769219 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.769294 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769278 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.769518 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769505 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.769562 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769508 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.769715 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769694 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Nov 04 01:56:08.769801 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769739 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Nov 04 01:56:08.769801 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769755 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Nov 04 01:56:08.769801 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769746 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.769801 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769762 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Nov 04 01:56:08.769801 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769773 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.769942 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.769871 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Nov 04 01:56:08.772575 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.772558 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/0a1d99ff-e629-46da-a1e0-e92fd128a237-config-volume\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.772641 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.772610 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/447db221-9de8-4e44-b228-ab76255a229d-konnectivity-ca\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:56:08.773701 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.773687 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.775354 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.775339 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-metrics-client-ca\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.776675 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.776653 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwbcd\" (UniqueName: \"kubernetes.io/projected/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-kube-api-access-fwbcd\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:56:08.776729 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.776665 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Nov 04 01:56:08.776765 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.776752 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kdrmc\" (UniqueName: \"kubernetes.io/projected/0a1d99ff-e629-46da-a1e0-e92fd128a237-kube-api-access-kdrmc\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.776845 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.776829 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xdjn\" (UniqueName: \"kubernetes.io/projected/636d2d6a-86f9-41ad-95f8-2b79b2645544-kube-api-access-7xdjn\") pod \"network-check-target-nkzgd\" (UID: \"636d2d6a-86f9-41ad-95f8-2b79b2645544\") " pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:08.783498 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783455 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/840c4e7c-424c-4f46-8f30-cb39f2d31a24-trusted-ca\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.783595 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783455 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h2vnh\" (UniqueName: \"kubernetes.io/projected/a3e06791-0981-4894-9ac1-d55aec5adf00-kube-api-access-h2vnh\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.783679 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783656 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bs62d\" (UniqueName: \"kubernetes.io/projected/078d2895-52e5-4a17-943a-ab3de7a3e40e-kube-api-access-bs62d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.783738 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783704 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8647w\" (UniqueName: \"kubernetes.io/projected/7cc1dde7-35d0-4bec-8ead-b06ecff9f71f-kube-api-access-8647w\") pod \"migrator-66df5d496b-zdttj\" (UID: \"7cc1dde7-35d0-4bec-8ead-b06ecff9f71f\") " pod="openshift-kube-storage-version-migrator/migrator-66df5d496b-zdttj" Nov 04 01:56:08.783798 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783739 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-skv9p\" (UniqueName: \"kubernetes.io/projected/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-kube-api-access-skv9p\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.784011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.783982 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbqdz\" (UniqueName: \"kubernetes.io/projected/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-kube-api-access-jbqdz\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.784082 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784000 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jksbb\" (UniqueName: \"kubernetes.io/projected/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kube-api-access-jksbb\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.784082 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784032 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hgnnc\" (UniqueName: \"kubernetes.io/projected/22ce2354-ad7a-4100-970f-ce84512cce41-kube-api-access-hgnnc\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.784082 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784045 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6qgt4\" (UniqueName: \"kubernetes.io/projected/f456ccdc-f27b-4026-b425-891f5c46263e-kube-api-access-6qgt4\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:56:08.784455 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784419 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-bound-sa-token\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.784547 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784515 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s8vbd\" (UniqueName: \"kubernetes.io/projected/ece54c3b-fb1f-4f03-9430-22f897a99c91-kube-api-access-s8vbd\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.784612 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784595 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2vxdx\" (UniqueName: \"kubernetes.io/projected/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-kube-api-access-2vxdx\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.784700 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l9b2\" (UniqueName: \"kubernetes.io/projected/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-kube-api-access-4l9b2\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.784992 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.784977 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqb6x\" (UniqueName: \"kubernetes.io/projected/0829f7ce-7f76-4727-bb67-7f60f08a3189-kube-api-access-wqb6x\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.785809 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785791 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.785870 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785824 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.785870 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785851 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785876 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785891 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-multus\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785900 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785893 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-tuning-conf-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785921 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-device-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785951 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-etc-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785956 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a3e06791-0981-4894-9ac1-d55aec5adf00-host-slash\") pod \"iptables-alerter-zd9fs\" (UID: \"a3e06791-0981-4894-9ac1-d55aec5adf00\") " pod="openshift-network-operator/iptables-alerter-zd9fs" Nov 04 01:56:08.786015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.785982 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786020 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786041 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786056 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786076 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786085 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786091 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-crio-socket\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786106 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-netns\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786111 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786142 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60-host\") pod \"node-ca-mbhp4\" (UID: \"f0735e53-cdcf-4cb6-b68c-10ed8eb1cd60\") " pod="openshift-image-registry/node-ca-mbhp4" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786140 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786164 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786180 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786179 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-kubelet\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786089 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-run-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786191 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysctl-conf\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786209 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786210 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-bin\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786236 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786259 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-modprobe-d\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786266 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786280 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-cnibin\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786260 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-systemd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786290 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786304 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-cnibin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786314 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786325 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-registration-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786340 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.786359 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786366 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-sys\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786382 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786400 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-netns\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786408 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786426 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-etc-kubernetes\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786436 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786460 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-kubelet-config\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786459 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786513 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-socket-dir-parent\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786519 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-conf-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786524 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786542 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-systemd-units\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786559 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786583 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786603 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786609 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786638 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786665 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786668 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-var-lib-kubelet\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786642 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-cni-netd\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786670 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-lib-modules\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786691 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786747 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38e16a3b-bcac-4482-b177-eb0061845a74-dbus\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786753 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786775 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-k8s-cni-cncf-io\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786691 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-run-multus-certs\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786782 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786802 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-cni-bin\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786814 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786824 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4tmtz\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-kube-api-access-4tmtz\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786842 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786871 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786884 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-host-var-lib-kubelet\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786897 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786902 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-kubelet-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786921 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786946 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786919 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-etc-selinux\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786979 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-kubernetes\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.786985 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787010 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-sysconfig\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787017 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787025 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-hostroot\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787027 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-multus-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787044 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787068 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787094 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787099 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-socket-dir\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787107 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7a827277-7c60-4dbd-b5a5-fc861a2c236d-sys-fs\") pod \"aws-ebs-csi-driver-node-slnfw\" (UID: \"7a827277-7c60-4dbd-b5a5-fc861a2c236d\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-slnfw" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787121 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-run-ovn\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787120 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787136 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0-hosts-file\") pod \"node-resolver-vdgnv\" (UID: \"ffc65ed1-fdd2-4112-bc05-a24ec76ec6b0\") " pod="openshift-dns/node-resolver-vdgnv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787150 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787165 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-host\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787171 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787186 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787187 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-var-lib-openvswitch\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787207 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-node-log\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787205 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787206 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/22ce2354-ad7a-4100-970f-ce84512cce41-system-cni-dir\") pod \"multus-additional-cni-plugins-59clt\" (UID: \"22ce2354-ad7a-4100-970f-ce84512cce41\") " pod="openshift-multus/multus-additional-cni-plugins-59clt" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787227 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-slash\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787238 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787256 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787270 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787283 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787289 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-log-socket\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787297 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787320 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/0829f7ce-7f76-4727-bb67-7f60f08a3189-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787335 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-run\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787333 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/078d2895-52e5-4a17-943a-ab3de7a3e40e-etc-systemd\") pod \"tuned-7s8zx\" (UID: \"078d2895-52e5-4a17-943a-ab3de7a3e40e\") " pod="openshift-cluster-node-tuning-operator/tuned-7s8zx" Nov 04 01:56:08.787510 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.787349 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ece54c3b-fb1f-4f03-9430-22f897a99c91-system-cni-dir\") pod \"multus-zxf47\" (UID: \"ece54c3b-fb1f-4f03-9430-22f897a99c91\") " pod="openshift-multus/multus-zxf47" Nov 04 01:56:08.806487 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806458 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Nov 04 01:56:08.806568 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806460 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Nov 04 01:56:08.806602 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806576 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrwjt\"" Nov 04 01:56:08.806602 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806582 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Nov 04 01:56:08.806680 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806665 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"ingress-canary-dockercfg-9ldb4\"" Nov 04 01:56:08.806709 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.806675 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-k89rs\"" Nov 04 01:56:08.807085 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807068 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Nov 04 01:56:08.807119 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807099 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-bzrbt\"" Nov 04 01:56:08.807119 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807098 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"global-pull-secret-syncer-dockercfg-wpjt7\"" Nov 04 01:56:08.807188 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807142 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-node-identity-dockercfg-zk98g\"" Nov 04 01:56:08.807188 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807143 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Nov 04 01:56:08.807188 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807180 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-c7hth\"" Nov 04 01:56:08.807264 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807243 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-zhvgj\"" Nov 04 01:56:08.807294 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807275 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-s4jjj\"" Nov 04 01:56:08.807327 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807321 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-7qkcb\"" Nov 04 01:56:08.807388 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807378 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Nov 04 01:56:08.807448 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807439 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-check-target-dockercfg-zzzll\"" Nov 04 01:56:08.807548 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807537 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-r7nnn\"" Nov 04 01:56:08.807607 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.807598 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-tvbnh\"" Nov 04 01:56:08.808079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.808067 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent-dockercfg-tlvkz\"" Nov 04 01:56:08.812203 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.812187 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-image-registry-private-configuration\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.813364 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.813351 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/0a1d99ff-e629-46da-a1e0-e92fd128a237-metrics-tls\") pod \"dns-default-p9lv2\" (UID: \"0a1d99ff-e629-46da-a1e0-e92fd128a237\") " pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:08.815541 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.815522 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.815583 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.815567 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/7299f7d4-b1ef-4a5d-a8fd-883550532d2c-prometheus-operator-tls\") pod \"prometheus-operator-9cd89b74c-ppdcg\" (UID: \"7299f7d4-b1ef-4a5d-a8fd-883550532d2c\") " pod="openshift-monitoring/prometheus-operator-9cd89b74c-ppdcg" Nov 04 01:56:08.815612 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.815581 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/0829f7ce-7f76-4727-bb67-7f60f08a3189-ovn-node-metrics-cert\") pod \"ovnkube-node-jm8wv\" (UID: \"0829f7ce-7f76-4727-bb67-7f60f08a3189\") " pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:08.818581 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.818538 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Nov 04 01:56:08.818927 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.818912 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Nov 04 01:56:08.819001 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.818931 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-mj9m5\"" Nov 04 01:56:08.819052 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819026 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-zsfkj\"" Nov 04 01:56:08.819052 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819034 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-bxxxw\"" Nov 04 01:56:08.819052 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819025 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Nov 04 01:56:08.819052 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819049 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Nov 04 01:56:08.819196 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819070 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Nov 04 01:56:08.819196 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819091 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-vn697\"" Nov 04 01:56:08.819196 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819092 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Nov 04 01:56:08.819196 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.819116 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Nov 04 01:56:08.823462 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.823446 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/bb2c7ac1-0806-4d8c-8c19-979d3c080f80-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-mrgm4\" (UID: \"bb2c7ac1-0806-4d8c-8c19-979d3c080f80\") " pod="openshift-insights/insights-runtime-extractor-mrgm4" Nov 04 01:56:08.823558 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.823461 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/840c4e7c-424c-4f46-8f30-cb39f2d31a24-registry-tls\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.823558 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.823536 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38e16a3b-bcac-4482-b177-eb0061845a74-original-pull-secret\") pod \"global-pull-secret-syncer-w2257\" (UID: \"38e16a3b-bcac-4482-b177-eb0061845a74\") " pod="kube-system/global-pull-secret-syncer-w2257" Nov 04 01:56:08.824608 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.824590 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/447db221-9de8-4e44-b228-ab76255a229d-agent-certs\") pod \"konnectivity-agent-75hc6\" (UID: \"447db221-9de8-4e44-b228-ab76255a229d\") " pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:56:08.824756 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.824742 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f456ccdc-f27b-4026-b425-891f5c46263e-cert\") pod \"ingress-canary-rjstr\" (UID: \"f456ccdc-f27b-4026-b425-891f5c46263e\") " pod="openshift-ingress-canary/ingress-canary-rjstr" Nov 04 01:56:08.825655 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.825638 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/840c4e7c-424c-4f46-8f30-cb39f2d31a24-installation-pull-secrets\") pod \"image-registry-75fb56bf5f-nt7k7\" (UID: \"840c4e7c-424c-4f46-8f30-cb39f2d31a24\") " pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:08.825723 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:08.825681 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a9dfb0df-5635-4cbd-81e6-6c55f20b3a17-metrics-certs\") pod \"network-metrics-daemon-sd84m\" (UID: \"a9dfb0df-5635-4cbd-81e6-6c55f20b3a17\") " pod="openshift-multus/network-metrics-daemon-sd84m" Nov 04 01:56:10.180735 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.180708 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:10.331713 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.331680 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p9lv2" Nov 04 01:56:10.333083 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.333054 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-2mst5"] Nov 04 01:56:10.335661 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.335646 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.388418 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.388389 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Nov 04 01:56:10.388660 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.388641 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Nov 04 01:56:10.388765 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.388754 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-hccdr\"" Nov 04 01:56:10.389222 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.389208 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Nov 04 01:56:10.398638 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398618 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-root\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398731 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398672 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-wtmp\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398730 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8v8l9\" (UniqueName: \"kubernetes.io/projected/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-kube-api-access-8v8l9\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398771 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-tls\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398928 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398911 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398989 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398936 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-textfile\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.398989 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.398959 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-accelerators-collector-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.399064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.399011 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-metrics-client-ca\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.399096 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.399067 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-sys\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.499905 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.499867 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-root\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.499925 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-wtmp\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.499945 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8v8l9\" (UniqueName: \"kubernetes.io/projected/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-kube-api-access-8v8l9\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.499964 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-tls\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.499992 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500010 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-textfile\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500031 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-accelerators-collector-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500064 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500051 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-metrics-client-ca\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500401 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500304 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-sys\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500459 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500402 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-sys\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500459 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500451 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-root\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500596 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500488 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-textfile\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.500596 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500543 5029 swap_util.go:78] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Nov 04 01:56:10.501034 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.500596 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-wtmp\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.501505 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.501268 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-accelerators-collector-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.501505 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.501274 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-metrics-client-ca\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.506461 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.506442 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.508395 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.508370 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-node-exporter-tls\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.561776 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.561739 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8v8l9\" (UniqueName: \"kubernetes.io/projected/ddb170ac-1121-452d-bf81-cef8e8e2e3d1-kube-api-access-8v8l9\") pod \"node-exporter-2mst5\" (UID: \"ddb170ac-1121-452d-bf81-cef8e8e2e3d1\") " pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.647258 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.647226 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-2mst5" Nov 04 01:56:10.658741 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:10.658708 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podddb170ac_1121_452d_bf81_cef8e8e2e3d1.slice/crio-f507ff98fa72da871351a65e88e55f88441a68a1482d0383e10eeafae362c043 WatchSource:0}: Error finding container f507ff98fa72da871351a65e88e55f88441a68a1482d0383e10eeafae362c043: Status 404 returned error can't find the container with id f507ff98fa72da871351a65e88e55f88441a68a1482d0383e10eeafae362c043 Nov 04 01:56:10.660849 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.660829 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 01:56:10.669872 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:10.669850 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2mst5" event={"ID":"ddb170ac-1121-452d-bf81-cef8e8e2e3d1","Type":"ContainerStarted","Data":"f507ff98fa72da871351a65e88e55f88441a68a1482d0383e10eeafae362c043"} Nov 04 01:56:11.688153 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:11.688121 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:56:12.679414 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:12.679376 5029 generic.go:356] "Generic (PLEG): container finished" podID="ddb170ac-1121-452d-bf81-cef8e8e2e3d1" containerID="f50de03fbc9d2691ffc22eb8f323668824640c3b8df8ba02e320ca8f38636357" exitCode=0 Nov 04 01:56:12.679592 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:12.679465 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2mst5" event={"ID":"ddb170ac-1121-452d-bf81-cef8e8e2e3d1","Type":"ContainerDied","Data":"f50de03fbc9d2691ffc22eb8f323668824640c3b8df8ba02e320ca8f38636357"} Nov 04 01:56:13.038124 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.038097 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:13.038502 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.038291 5029 prober_manager.go:356] "Failed to trigger a manual run" probe="Readiness" Nov 04 01:56:13.685609 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.685576 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2mst5" event={"ID":"ddb170ac-1121-452d-bf81-cef8e8e2e3d1","Type":"ContainerStarted","Data":"a012ed51cde3d074286645143062dc1aa5ff94827ce02eaa6f96c0e7e010c023"} Nov 04 01:56:13.685780 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.685628 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-2mst5" event={"ID":"ddb170ac-1121-452d-bf81-cef8e8e2e3d1","Type":"ContainerStarted","Data":"6ec66a17a1cd8250d6d84789f823b41a92b1082ee3ff532c936a1d34d20daad3"} Nov 04 01:56:13.697326 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.697300 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jm8wv" Nov 04 01:56:13.815922 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:13.815857 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-2mst5" podStartSLOduration=2.460473227 podStartE2EDuration="3.815840242s" totalImagesPullingTime="1.280157899s" totalInitContainerRuntime="75.209116ms" isStatefulPod=true podCreationTimestamp="2026-11-04 01:56:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:13.814939165 +0000 UTC m=+6.308264305" watchObservedRunningTime="2026-11-04 01:56:13.815840242 +0000 UTC m=+6.309165384" Nov 04 01:56:14.941357 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:14.941325 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-75fb56bf5f-nt7k7" Nov 04 01:56:18.127728 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:18.127655 5029 kubelet_node_status.go:683] "Recording event message for node" node="ip-10-0-134-176.us-west-1.compute.internal" event="NodeReady" Nov 04 01:56:19.655942 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.655905 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:56:19.658339 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.658319 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.710724 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710683 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 01:56:19.710724 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710683 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 01:56:19.710921 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710726 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 01:56:19.710921 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710736 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 01:56:19.710921 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710889 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 01:56:19.710921 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.710911 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 01:56:19.760408 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.760374 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 01:56:19.764394 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764370 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764462 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764407 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68vhf\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764462 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764428 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764462 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764450 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764592 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764503 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764592 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764570 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764683 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764593 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764683 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764610 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764683 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764630 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764832 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764706 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764832 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764769 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764904 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764827 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764904 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764868 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.764904 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764901 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765010 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764922 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765010 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.764948 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765010 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.765002 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765119 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.765019 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765119 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.765040 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.765119 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.765054 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.811213 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.811178 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 01:56:19.811372 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.811244 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 01:56:19.811591 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.811574 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 01:56:19.811942 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.811925 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:56:19.860914 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860891 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860936 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860949 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7k9go77f3i4r1\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860962 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860954 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.860985 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 01:56:19.861075 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.861023 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6k9s5\"" Nov 04 01:56:19.866144 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866127 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866152 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866174 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866295 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866192 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866350 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866319 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866403 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866362 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866403 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866391 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866508 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866418 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866508 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866445 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866508 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866499 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866654 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866544 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866654 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866579 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866654 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866603 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866654 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866633 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.866654 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.866636 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867061 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867039 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867184 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867078 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867184 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867102 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867184 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867134 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867184 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867167 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-68vhf\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867446 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867203 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867446 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867235 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867446 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867435 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.867860 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.867785 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.868088 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.868064 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.868415 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.868385 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.868538 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.868490 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.870611 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.870571 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.871461 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.871438 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.871576 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.871459 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.871576 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.871447 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.871689 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.871584 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.872283 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.872258 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.872338 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.872291 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.872464 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.872442 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.873089 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.873059 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.873174 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.873145 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.873216 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.873173 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.873657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.873642 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.929719 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.929653 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68vhf\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\") pod \"prometheus-k8s-0\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:19.972535 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:19.972507 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:20.406047 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:20.406009 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:56:20.408904 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:56:20.408882 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40261a3_7702_43c8_811c_f5442f8bda1b.slice/crio-3045cd40769ec981b2c7d4d0468d3b1dc01facae5ef46f9b96f2520d21436fb6 WatchSource:0}: Error finding container 3045cd40769ec981b2c7d4d0468d3b1dc01facae5ef46f9b96f2520d21436fb6: Status 404 returned error can't find the container with id 3045cd40769ec981b2c7d4d0468d3b1dc01facae5ef46f9b96f2520d21436fb6 Nov 04 01:56:20.708350 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:20.708249 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"3045cd40769ec981b2c7d4d0468d3b1dc01facae5ef46f9b96f2520d21436fb6"} Nov 04 01:56:22.716267 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:22.716235 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" exitCode=0 Nov 04 01:56:22.716682 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:22.716301 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} Nov 04 01:56:26.731633 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:26.731599 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} Nov 04 01:56:26.732018 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:26.731675 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} Nov 04 01:56:28.754840 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:28.754697 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} Nov 04 01:56:28.754840 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:28.754779 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} Nov 04 01:56:29.761689 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:29.761653 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} Nov 04 01:56:29.762094 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:29.761713 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerStarted","Data":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} Nov 04 01:56:29.973079 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:29.973030 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:56:30.041181 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:30.041068 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.769188222 podStartE2EDuration="11.041044457s" totalImagesPullingTime="7.253578928s" totalInitContainerRuntime="18.277307ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:56:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:56:29.991213813 +0000 UTC m=+22.484538953" watchObservedRunningTime="2026-11-04 01:56:30.041044457 +0000 UTC m=+22.534369592" Nov 04 01:56:34.501622 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:56:34.501581 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:56:34.717111 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:34.717078 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-75hc6" Nov 04 01:56:41.690877 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:56:41.690850 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nkzgd" Nov 04 01:57:04.501235 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:04.501194 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:57:20.516543 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:20.516505 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:20.535680 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:20.535650 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:20.943017 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:20.942948 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:34.501781 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:34.501745 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:57:42.919491 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919435 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:42.919890 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919831 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="prometheus" containerID="cri-o://73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" gracePeriod=600 Nov 04 01:57:42.919965 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919851 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy" containerID="cri-o://b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" gracePeriod=600 Nov 04 01:57:42.919965 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919915 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="thanos-sidecar" containerID="cri-o://42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" gracePeriod=600 Nov 04 01:57:42.920080 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919952 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-web" containerID="cri-o://f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" gracePeriod=600 Nov 04 01:57:42.920080 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919957 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-thanos" containerID="cri-o://ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" gracePeriod=600 Nov 04 01:57:42.920080 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:42.919970 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="config-reloader" containerID="cri-o://68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" gracePeriod=600 Nov 04 01:57:43.188542 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.188519 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:43.336602 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336568 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336615 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336636 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336674 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336696 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336718 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336748 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.336795 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336773 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336809 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336841 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336869 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336898 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.336943 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337000 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337045 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337072 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337103 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337137 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337159 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337194 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337200 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.337824 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337229 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\") pod \"c40261a3-7702-43c8-811c-f5442f8bda1b\" (UID: \"c40261a3-7702-43c8-811c-f5442f8bda1b\") " Nov 04 01:57:43.340234 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337426 5029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.340234 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.337682 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.340234 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.338422 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.340234 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.338665 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.340234 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.338759 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.340728 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.340691 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 01:57:43.342712 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.342686 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.342916 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.342896 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "kube-api-access-68vhf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 01:57:43.343350 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343323 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.343533 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343507 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 01:57:43.343673 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343568 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.343939 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343858 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.343939 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343914 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.344092 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.343885 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 01:57:43.344354 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.344328 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.344783 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.344759 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 01:57:43.345524 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.345501 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.345610 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.345545 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.345610 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.345591 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.353972 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.353951 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config" pod "c40261a3-7702-43c8-811c-f5442f8bda1b" (UID: "c40261a3-7702-43c8-811c-f5442f8bda1b"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440607 5029 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-kube-rbac-proxy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440639 5029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440653 5029 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-config\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440665 5029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-db\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440677 5029 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-metrics-client-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.440693 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440689 5029 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-web-config\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440703 5029 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440718 5029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440730 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-68vhf\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-kube-api-access-68vhf\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440741 5029 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440753 5029 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-grpc-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440764 5029 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-thanos-prometheus-http-client-file\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440777 5029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440789 5029 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/c40261a3-7702-43c8-811c-f5442f8bda1b-config-out\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440802 5029 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-prometheus-trusted-ca-bundle\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440814 5029 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c40261a3-7702-43c8-811c-f5442f8bda1b-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440827 5029 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-metrics-client-ca\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440839 5029 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c40261a3-7702-43c8-811c-f5442f8bda1b-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:43.441011 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:43.440851 5029 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/c40261a3-7702-43c8-811c-f5442f8bda1b-tls-assets\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 01:57:44.004211 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004171 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" exitCode=0 Nov 04 01:57:44.004211 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004201 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" exitCode=0 Nov 04 01:57:44.004211 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004210 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" exitCode=0 Nov 04 01:57:44.004211 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004218 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" exitCode=0 Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004227 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" exitCode=0 Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004234 5029 generic.go:356] "Generic (PLEG): container finished" podID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" exitCode=0 Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004248 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004242 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004380 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004406 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004432 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004450 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004482 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004521 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} Nov 04 01:57:44.004787 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.004547 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"c40261a3-7702-43c8-811c-f5442f8bda1b","Type":"ContainerDied","Data":"3045cd40769ec981b2c7d4d0468d3b1dc01facae5ef46f9b96f2520d21436fb6"} Nov 04 01:57:44.012045 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.012029 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.018483 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.018456 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.024851 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.024835 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.031224 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.031208 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.037370 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.037349 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.044369 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.044350 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.053355 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.053336 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.053639 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.053619 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.053706 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.053652 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.053706 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.053696 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.053973 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.053952 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.054015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.053980 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.054015 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.053995 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.054228 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.054211 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.054265 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054234 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.054265 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054250 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.054461 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.054445 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.054544 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054489 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.054544 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054508 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.054758 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.054743 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.054809 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054761 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.054809 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054773 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.054991 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.054973 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.055039 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.054996 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.055039 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055010 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.055217 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:57:44.055201 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.055289 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055224 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.055289 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055244 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.055507 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055490 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.055559 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055517 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.055717 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055700 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.055759 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055716 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.055902 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055884 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.055938 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.055902 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.056127 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056108 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.056195 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056128 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.056353 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056335 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.056397 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056352 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.056580 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056558 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.056580 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056578 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.056774 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056756 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.056826 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056774 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.057006 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.056986 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.057066 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057008 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.057233 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057218 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.057233 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057232 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.057464 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057442 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.057576 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057463 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.057736 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057716 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.057788 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057737 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.057981 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057962 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.058017 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.057981 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.058200 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058183 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.058242 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058201 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.058419 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058399 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.058482 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058420 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.058661 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058643 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.058711 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058661 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.058867 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058851 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.058867 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.058866 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.059069 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059052 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.059107 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059070 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.059309 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059292 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.059350 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059310 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.059538 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059522 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.059586 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059538 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.059737 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059721 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.059773 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059737 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.059931 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059912 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.059972 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.059931 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.060141 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060126 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.060141 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060140 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.060357 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060339 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.060357 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060357 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.060589 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060569 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.060589 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060589 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.060813 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060791 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.060857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.060813 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.061039 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061022 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.061039 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061039 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.061270 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061252 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.061270 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061269 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.061516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061499 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.061516 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061515 5029 scope.go:122] "RemoveContainer" containerID="ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4" Nov 04 01:57:44.061751 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061731 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4"} err="failed to get container status \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": rpc error: code = NotFound desc = could not find container \"ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4\": container with ID starting with ff8947538b92ce79524311414244c943456461e93e960455ffe8f273427cfdc4 not found: ID does not exist" Nov 04 01:57:44.061802 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061752 5029 scope.go:122] "RemoveContainer" containerID="b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4" Nov 04 01:57:44.061979 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061959 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4"} err="failed to get container status \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": rpc error: code = NotFound desc = could not find container \"b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4\": container with ID starting with b65d3f8ce90435034bfd9f3dc367b490299da48053f3e588479628ec4933a8e4 not found: ID does not exist" Nov 04 01:57:44.062020 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.061980 5029 scope.go:122] "RemoveContainer" containerID="f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9" Nov 04 01:57:44.062174 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062159 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9"} err="failed to get container status \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": rpc error: code = NotFound desc = could not find container \"f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9\": container with ID starting with f2dfda31cbfb63b6d2474805738eedbca5ba099be6ab47540c9d85ccaf8132d9 not found: ID does not exist" Nov 04 01:57:44.062174 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062173 5029 scope.go:122] "RemoveContainer" containerID="42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1" Nov 04 01:57:44.062385 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062368 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1"} err="failed to get container status \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": rpc error: code = NotFound desc = could not find container \"42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1\": container with ID starting with 42779bf8036e69b48693771a65259ae30326899a18c46849a71e9eb9bbd0c4a1 not found: ID does not exist" Nov 04 01:57:44.062424 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062385 5029 scope.go:122] "RemoveContainer" containerID="68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3" Nov 04 01:57:44.062609 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062593 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3"} err="failed to get container status \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": rpc error: code = NotFound desc = could not find container \"68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3\": container with ID starting with 68816fe2b5680da1d0785176db0442b75068ee1eb185f87cc97ecf06f01f11c3 not found: ID does not exist" Nov 04 01:57:44.062663 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062609 5029 scope.go:122] "RemoveContainer" containerID="73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310" Nov 04 01:57:44.062839 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062821 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310"} err="failed to get container status \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": rpc error: code = NotFound desc = could not find container \"73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310\": container with ID starting with 73b39e08eddee18a8aa547ab486f1ca3fa79b71364a7c6dbfc5e5658eb7b8310 not found: ID does not exist" Nov 04 01:57:44.062839 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.062838 5029 scope.go:122] "RemoveContainer" containerID="ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e" Nov 04 01:57:44.063055 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.063037 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e"} err="failed to get container status \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": rpc error: code = NotFound desc = could not find container \"ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e\": container with ID starting with ca6070d0f6a79710753fe71bf1e553eb5ffcc01c7119c4e68e0a7a920cfad35e not found: ID does not exist" Nov 04 01:57:44.171728 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.171702 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:44.177859 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.177840 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:44.221819 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.221732 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:44.222081 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222063 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="init-config-reloader" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222083 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="init-config-reloader" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222103 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="config-reloader" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222112 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="config-reloader" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222126 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-web" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222133 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-web" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222143 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy" Nov 04 01:57:44.222151 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222151 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222164 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="thanos-sidecar" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222171 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="thanos-sidecar" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222181 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-thanos" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222190 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-thanos" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222203 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="prometheus" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222210 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="prometheus" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222265 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-web" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222277 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222287 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="kube-rbac-proxy-thanos" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222295 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="prometheus" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222305 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="config-reloader" Nov 04 01:57:44.222514 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.222314 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" containerName="thanos-sidecar" Nov 04 01:57:44.225600 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.225581 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.279155 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.279127 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Nov 04 01:57:44.279283 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.279161 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Nov 04 01:57:44.279283 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.279185 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Nov 04 01:57:44.280063 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280039 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Nov 04 01:57:44.280063 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280051 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Nov 04 01:57:44.280063 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280062 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Nov 04 01:57:44.280235 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280102 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Nov 04 01:57:44.280235 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280162 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Nov 04 01:57:44.280235 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280168 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Nov 04 01:57:44.280235 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280218 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Nov 04 01:57:44.280523 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280508 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-6k9s5\"" Nov 04 01:57:44.280523 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280511 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Nov 04 01:57:44.280617 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.280523 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7k9go77f3i4r1\"" Nov 04 01:57:44.328735 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.328713 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Nov 04 01:57:44.330167 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.330149 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Nov 04 01:57:44.330279 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.330243 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Nov 04 01:57:44.330339 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.330290 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Nov 04 01:57:44.348054 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348030 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348153 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348056 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348153 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348077 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348153 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348102 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-web-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348254 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348166 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348254 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348192 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348254 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348243 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348338 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348261 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348338 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348281 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348397 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348359 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348397 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348376 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348397 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348393 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348408 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348424 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348522 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348461 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348641 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348533 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348641 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348569 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348641 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348595 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348786 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348659 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config-out\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.348786 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.348701 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kpzh6\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-kube-api-access-kpzh6\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449435 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449395 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449443 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449500 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449525 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config-out\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449657 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449551 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kpzh6\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-kube-api-access-kpzh6\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449712 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449758 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449790 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449827 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-web-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.449857 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449855 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449879 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449915 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449941 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.449982 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450017 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450041 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450067 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450095 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450090 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450770 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450117 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450770 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450140 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.450770 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.450595 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.451124 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.451099 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.451210 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.451190 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.451573 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.451548 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.451773 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.451747 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.452077 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.452052 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.452793 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.452769 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.453328 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.453301 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.453929 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.453888 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config-out\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.454660 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.454637 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.454926 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.454903 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.455022 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.455009 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.455658 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.455632 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.455736 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.455700 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.455736 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.455717 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.456107 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.456084 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.456107 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.456095 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.456276 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.456256 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-web-config\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.456896 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.456881 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.465758 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.465711 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:44.510916 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.510882 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kpzh6\" (UniqueName: \"kubernetes.io/projected/04c8d696-35f9-4ade-ac3b-9bb205e7d34a-kube-api-access-kpzh6\") pod \"prometheus-k8s-0\" (UID: \"04c8d696-35f9-4ade-ac3b-9bb205e7d34a\") " pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.540782 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.540757 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:57:44.970672 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:44.970608 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Nov 04 01:57:44.974030 ip-10-0-134-176 kubenswrapper[5029]: W1104 01:57:44.974003 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04c8d696_35f9_4ade_ac3b_9bb205e7d34a.slice/crio-e1d13e74160f19bdd4c90009781890caccafb36ee8c80a66d2b401331bfa97f0 WatchSource:0}: Error finding container e1d13e74160f19bdd4c90009781890caccafb36ee8c80a66d2b401331bfa97f0: Status 404 returned error can't find the container with id e1d13e74160f19bdd4c90009781890caccafb36ee8c80a66d2b401331bfa97f0 Nov 04 01:57:45.010153 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:45.010126 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"e1d13e74160f19bdd4c90009781890caccafb36ee8c80a66d2b401331bfa97f0"} Nov 04 01:57:45.617526 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:45.617495 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="c40261a3-7702-43c8-811c-f5442f8bda1b" path="/var/lib/kubelet/pods/c40261a3-7702-43c8-811c-f5442f8bda1b/volumes" Nov 04 01:57:46.015225 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:46.015145 5029 generic.go:356] "Generic (PLEG): container finished" podID="04c8d696-35f9-4ade-ac3b-9bb205e7d34a" containerID="a9ed58cf6a3f775dad40b53cb09f500ebfd4f6223488dbb126fb278fe8ed9c74" exitCode=0 Nov 04 01:57:46.015225 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:46.015182 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerDied","Data":"a9ed58cf6a3f775dad40b53cb09f500ebfd4f6223488dbb126fb278fe8ed9c74"} Nov 04 01:57:47.022600 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022560 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"dbfe5e9fac111a0a7608fc17f94214b400a67304a32d2b1265b483182934d38b"} Nov 04 01:57:47.022985 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022650 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"b2ca1151037d23da246b4b1dbe2f63e2de67500092d99cac18e1f29a922012c1"} Nov 04 01:57:47.022985 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022677 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"f49ca5137b3b7dfc00f495f68a431310a77ec061e5b48f70e609c35edf67f833"} Nov 04 01:57:47.022985 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022702 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"1a1f4069bca8ecbc52b3598f464114345fa5e07c4491f5bda1a0bf617e593a84"} Nov 04 01:57:47.022985 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022724 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"0645a98e340699c4b557d280de20e12dd9547f7d04f7150dd59985b12aa26032"} Nov 04 01:57:47.022985 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.022746 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"04c8d696-35f9-4ade-ac3b-9bb205e7d34a","Type":"ContainerStarted","Data":"c03ed8d3ce307fbdb1d0bc0c1a2c454453709e7f329bc3c0dfd92f93c19e828e"} Nov 04 01:57:47.158490 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:47.158407 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.143732761 podStartE2EDuration="3.158391525s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.658764ms" isStatefulPod=false podCreationTimestamp="2026-11-04 01:57:44 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 01:57:47.158162626 +0000 UTC m=+99.651488005" watchObservedRunningTime="2026-11-04 01:57:47.158391525 +0000 UTC m=+99.651716711" Nov 04 01:57:49.541270 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:57:49.541223 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:58:04.500938 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:58:04.500902 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:58:34.501051 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:58:34.501009 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:58:44.541843 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:58:44.541794 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:58:44.557546 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:58:44.557523 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:58:45.227324 ip-10-0-134-176 kubenswrapper[5029]: I1104 01:58:45.227293 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Nov 04 01:59:04.502514 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:59:04.502456 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 01:59:34.501818 ip-10-0-134-176 kubenswrapper[5029]: E1104 01:59:34.501776 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:00:04.501688 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:00:04.501648 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:00:06.142213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.142180 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds"] Nov 04 02:00:06.145532 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.145500 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.174133 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.174109 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.174283 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.174159 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vt2v4\" (UniqueName: \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.174361 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.174273 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.198461 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.198432 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Nov 04 02:00:06.248256 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.248220 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Nov 04 02:00:06.248405 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.248275 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"e3d011e69d9573cabb8c359e5ac709eb5ac240-be78d160-dockercfg-mzqkb\"" Nov 04 02:00:06.248899 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.248881 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds"] Nov 04 02:00:06.274639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.274613 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vt2v4\" (UniqueName: \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.274732 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.274663 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.274800 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.274780 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.275070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.275051 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.275107 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.275066 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.335754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.335729 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vt2v4\" (UniqueName: \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\") pod \"e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.460011 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.459932 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:06.830048 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:06.830011 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds"] Nov 04 02:00:06.833256 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:00:06.833230 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc556cb20_cbb7_48a0_9ad2_dbe08a638f3a.slice/crio-483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5 WatchSource:0}: Error finding container 483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5: Status 404 returned error can't find the container with id 483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5 Nov 04 02:00:07.467507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:07.467453 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerStarted","Data":"483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5"} Nov 04 02:00:13.491329 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:13.491294 5029 generic.go:356] "Generic (PLEG): container finished" podID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerID="e7d3afb796151421d30b2f06a1e68ea87008eecab0a900305f5ce7f6d13c98a6" exitCode=0 Nov 04 02:00:13.491742 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:13.491341 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerDied","Data":"e7d3afb796151421d30b2f06a1e68ea87008eecab0a900305f5ce7f6d13c98a6"} Nov 04 02:00:19.513504 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:19.513454 5029 generic.go:356] "Generic (PLEG): container finished" podID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerID="b5a3e217a64fd52f8a7a1d3b6e34cd8041b1518896865b81875afc6dd621ed6a" exitCode=0 Nov 04 02:00:19.513914 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:19.513518 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerDied","Data":"b5a3e217a64fd52f8a7a1d3b6e34cd8041b1518896865b81875afc6dd621ed6a"} Nov 04 02:00:27.542950 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:27.542909 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerStarted","Data":"59d8bec44ac5980e30f9da546c0a9a061adf662825452794d860e00429a2d8c8"} Nov 04 02:00:27.660009 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:27.659903 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" podStartSLOduration=2.910836271 podStartE2EDuration="21.659891135s" totalImagesPullingTime="18.749054864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:27.659875923 +0000 UTC m=+260.153201388" watchObservedRunningTime="2026-11-04 02:00:27.659891135 +0000 UTC m=+260.153216270" Nov 04 02:00:28.548530 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:28.548495 5029 generic.go:356] "Generic (PLEG): container finished" podID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerID="59d8bec44ac5980e30f9da546c0a9a061adf662825452794d860e00429a2d8c8" exitCode=0 Nov 04 02:00:28.548931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:28.548581 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerDied","Data":"59d8bec44ac5980e30f9da546c0a9a061adf662825452794d860e00429a2d8c8"} Nov 04 02:00:29.697488 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.697451 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:29.767190 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.767159 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\") pod \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " Nov 04 02:00:29.767321 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.767218 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\" (UniqueName: \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\") pod \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " Nov 04 02:00:29.767321 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.767249 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\") pod \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\" (UID: \"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a\") " Nov 04 02:00:29.767823 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.767782 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle" pod "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" (UID: "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:00:29.769656 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.769628 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4" pod "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" (UID: "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a"). InnerVolumeSpecName "kube-api-access-vt2v4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:00:29.771758 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.771738 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util" pod "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" (UID: "c556cb20-cbb7-48a0-9ad2-dbe08a638f3a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:00:29.867825 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.867747 5029 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-util\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:00:29.867825 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.867776 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vt2v4\" (UniqueName: \"kubernetes.io/projected/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-kube-api-access-vt2v4\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:00:29.867825 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:29.867786 5029 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c556cb20-cbb7-48a0-9ad2-dbe08a638f3a-bundle\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:00:30.555951 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:30.555916 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" event={"ID":"c556cb20-cbb7-48a0-9ad2-dbe08a638f3a","Type":"ContainerDied","Data":"483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5"} Nov 04 02:00:30.555951 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:30.555947 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/e3d011e69d9573cabb8c359e5ac709eb5ac2407647b5169ad26d957e4c26lds" Nov 04 02:00:30.556162 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:30.555969 5029 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="483e5ec3eb8a13f4de1bb56a84bb783e23afd3d2f2b89d074483d619c2cdc1c5" Nov 04 02:00:34.501395 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:00:34.501355 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:00:47.065209 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065131 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld"] Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065373 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="extract" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065383 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="extract" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065393 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="pull" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065398 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="pull" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065423 5029 cpu_manager.go:414] "containerMap: removing container" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="util" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065429 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="util" Nov 04 02:00:47.065613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.065481 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="c556cb20-cbb7-48a0-9ad2-dbe08a638f3a" containerName="extract" Nov 04 02:00:47.067038 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.067023 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.119906 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.119882 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"kube-root-ca.crt\"" Nov 04 02:00:47.119906 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.119881 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-webhook-cert\"" Nov 04 02:00:47.120136 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.120121 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-dockercfg-2szzx\"" Nov 04 02:00:47.120235 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.120220 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-controller-manager-service-cert\"" Nov 04 02:00:47.169763 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.169709 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub-operator-system\"/\"openshift-service-ca.crt\"" Nov 04 02:00:47.169905 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.169799 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub-operator-system\"/\"opendatahub-operator-metrics-tls\"" Nov 04 02:00:47.171451 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.171431 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld"] Nov 04 02:00:47.195194 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195168 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.195266 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195198 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.195266 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195228 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.195401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195388 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.195437 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195409 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thjdr\" (UniqueName: \"kubernetes.io/projected/c65d4bb8-da39-4175-9697-d920190a9054-kube-api-access-thjdr\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.195498 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.195440 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296502 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296452 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296518 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296536 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thjdr\" (UniqueName: \"kubernetes.io/projected/c65d4bb8-da39-4175-9697-d920190a9054-kube-api-access-thjdr\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296561 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296582 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296599 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.296953 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.296931 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-manifests\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.299156 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.299132 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-metrics-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.299265 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.299171 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c65d4bb8-da39-4175-9697-d920190a9054-tmp\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.299265 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.299256 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-webhook-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.299409 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.299387 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/c65d4bb8-da39-4175-9697-d920190a9054-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.362750 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.362685 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-thjdr\" (UniqueName: \"kubernetes.io/projected/c65d4bb8-da39-4175-9697-d920190a9054-kube-api-access-thjdr\") pod \"opendatahub-operator-controller-manager-7f9fcc96f6-p2xld\" (UID: \"c65d4bb8-da39-4175-9697-d920190a9054\") " pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.378541 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.378515 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:47.867247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:47.867216 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld"] Nov 04 02:00:47.870427 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:00:47.870402 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc65d4bb8_da39_4175_9697_d920190a9054.slice/crio-ce22124ae0a636f3edab7ace7d8989ec8dbc96dbb6f223da1d14f3307c217585 WatchSource:0}: Error finding container ce22124ae0a636f3edab7ace7d8989ec8dbc96dbb6f223da1d14f3307c217585: Status 404 returned error can't find the container with id ce22124ae0a636f3edab7ace7d8989ec8dbc96dbb6f223da1d14f3307c217585 Nov 04 02:00:48.616451 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:48.616414 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" event={"ID":"c65d4bb8-da39-4175-9697-d920190a9054","Type":"ContainerStarted","Data":"ce22124ae0a636f3edab7ace7d8989ec8dbc96dbb6f223da1d14f3307c217585"} Nov 04 02:00:53.635098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:53.635063 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" event={"ID":"c65d4bb8-da39-4175-9697-d920190a9054","Type":"ContainerStarted","Data":"ec1744c59c284d580e0bab4e11faf87d4f53c9e3fbdf282879d5b029cd6d1f59"} Nov 04 02:00:54.640000 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:54.639964 5029 generic.go:356] "Generic (PLEG): container finished" podID="c65d4bb8-da39-4175-9697-d920190a9054" containerID="ec1744c59c284d580e0bab4e11faf87d4f53c9e3fbdf282879d5b029cd6d1f59" exitCode=0 Nov 04 02:00:54.640000 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:54.640001 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" event={"ID":"c65d4bb8-da39-4175-9697-d920190a9054","Type":"ContainerDied","Data":"ec1744c59c284d580e0bab4e11faf87d4f53c9e3fbdf282879d5b029cd6d1f59"} Nov 04 02:00:55.644405 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:55.644372 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" event={"ID":"c65d4bb8-da39-4175-9697-d920190a9054","Type":"ContainerStarted","Data":"0a3bb3d288c2edc3bb6ca03c52de700878df9dea421063cee42fccc3bff1cdd0"} Nov 04 02:00:55.644744 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:55.644499 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:00:55.785925 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:00:55.785817 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" podStartSLOduration=3.208589187 podStartE2EDuration="9.785801171s" totalImagesPullingTime="6.158619278s" totalInitContainerRuntime="418.592706ms" isStatefulPod=false podCreationTimestamp="2026-11-04 02:00:46 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:00:55.784488997 +0000 UTC m=+288.277814132" watchObservedRunningTime="2026-11-04 02:00:55.785801171 +0000 UTC m=+288.279126315" Nov 04 02:01:00.795563 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.795525 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl"] Nov 04 02:01:00.799029 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.799007 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:00.858576 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.858548 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl"] Nov 04 02:01:00.889922 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.889893 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wdr67\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-kube-api-access-wdr67\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:00.890050 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.889934 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:00.903799 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.903769 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Nov 04 02:01:00.903799 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.903774 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Nov 04 02:01:00.998491 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.997778 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wdr67\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-kube-api-access-wdr67\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:00.998491 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:00.997833 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:01.059603 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:01.059539 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wdr67\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-kube-api-access-wdr67\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:01.060430 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:01.060415 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/1b259919-872a-45fb-b69a-f14433ca4a1a-bound-sa-token\") pod \"cert-manager-webhook-5dbbbf75cc-xcpfl\" (UID: \"1b259919-872a-45fb-b69a-f14433ca4a1a\") " pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:01.120801 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:01.120780 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:01.507316 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:01.507288 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl"] Nov 04 02:01:01.510408 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:01.510380 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1b259919_872a_45fb_b69a_f14433ca4a1a.slice/crio-1a344d1f1d98cbc165e0b809ce4ffc4dd4e30ccef5e57c7c7dcad647731cebec WatchSource:0}: Error finding container 1a344d1f1d98cbc165e0b809ce4ffc4dd4e30ccef5e57c7c7dcad647731cebec: Status 404 returned error can't find the container with id 1a344d1f1d98cbc165e0b809ce4ffc4dd4e30ccef5e57c7c7dcad647731cebec Nov 04 02:01:01.662408 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:01.662382 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" event={"ID":"1b259919-872a-45fb-b69a-f14433ca4a1a","Type":"ContainerStarted","Data":"1a344d1f1d98cbc165e0b809ce4ffc4dd4e30ccef5e57c7c7dcad647731cebec"} Nov 04 02:01:04.507775 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:01:04.507732 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:01:06.649417 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:06.649385 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub-operator-system/opendatahub-operator-controller-manager-7f9fcc96f6-p2xld" Nov 04 02:01:06.682132 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:06.682099 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" event={"ID":"1b259919-872a-45fb-b69a-f14433ca4a1a","Type":"ContainerStarted","Data":"e730d6341f6ed1c3258fe0cfd63535f4b334602b092ffc4280f2037acbc68cf1"} Nov 04 02:01:06.682305 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:06.682226 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:07.007413 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.007365 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" podStartSLOduration=2.41681132 podStartE2EDuration="7.007351539s" totalImagesPullingTime="4.590540219s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:07.006462457 +0000 UTC m=+299.499787601" watchObservedRunningTime="2026-11-04 02:01:07.007351539 +0000 UTC m=+299.500676681" Nov 04 02:01:07.421091 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.421006 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv"] Nov 04 02:01:07.423057 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.423035 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.452509 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.452461 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2dpv\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-kube-api-access-f2dpv\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.452616 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.452581 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.476282 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.476260 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ckvsp\"" Nov 04 02:01:07.483013 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.482991 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv"] Nov 04 02:01:07.553205 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.553179 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2dpv\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-kube-api-access-f2dpv\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.553307 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.553247 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.618231 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.618204 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-bound-sa-token\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.618331 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.618245 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2dpv\" (UniqueName: \"kubernetes.io/projected/95a7ed3e-6212-4dc4-87fe-4252cc3cce48-kube-api-access-f2dpv\") pod \"cert-manager-cainjector-5c57bbd4fc-r9wmv\" (UID: \"95a7ed3e-6212-4dc4-87fe-4252cc3cce48\") " pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:07.784790 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.784757 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ckvsp\"" Nov 04 02:01:07.792798 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:07.792779 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" Nov 04 02:01:08.166415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:08.166336 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv"] Nov 04 02:01:08.169606 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:08.169574 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95a7ed3e_6212_4dc4_87fe_4252cc3cce48.slice/crio-85e310b4d4d6c789ed137ce854a6911b29b9a84889ad881a26a714d8166d8beb WatchSource:0}: Error finding container 85e310b4d4d6c789ed137ce854a6911b29b9a84889ad881a26a714d8166d8beb: Status 404 returned error can't find the container with id 85e310b4d4d6c789ed137ce854a6911b29b9a84889ad881a26a714d8166d8beb Nov 04 02:01:08.690797 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:08.690763 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" event={"ID":"95a7ed3e-6212-4dc4-87fe-4252cc3cce48","Type":"ContainerStarted","Data":"1fcc90d226903b2e69b39ee4348cd6e43638835ed085acf83491217ad34083bb"} Nov 04 02:01:08.690797 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:08.690806 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" event={"ID":"95a7ed3e-6212-4dc4-87fe-4252cc3cce48","Type":"ContainerStarted","Data":"85e310b4d4d6c789ed137ce854a6911b29b9a84889ad881a26a714d8166d8beb"} Nov 04 02:01:08.815909 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:08.815847 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-5c57bbd4fc-r9wmv" podStartSLOduration=1.8158305989999999 podStartE2EDuration="1.815830599s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:08.815651911 +0000 UTC m=+301.308977063" watchObservedRunningTime="2026-11-04 02:01:08.815830599 +0000 UTC m=+301.309155740" Nov 04 02:01:12.687043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:12.687011 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-5dbbbf75cc-xcpfl" Nov 04 02:01:23.069631 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.069592 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7"] Nov 04 02:01:23.072700 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.072682 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:23.130650 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.130613 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Nov 04 02:01:23.130650 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.130633 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Nov 04 02:01:23.130861 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.130683 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Nov 04 02:01:23.167131 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.167095 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pw9t7\" (UniqueName: \"kubernetes.io/projected/0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d-kube-api-access-pw9t7\") pod \"dns-operator-controller-manager-8557cbdb84-nfhp7\" (UID: \"0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:23.180100 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.180076 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-ldfgr\"" Nov 04 02:01:23.181129 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.181110 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7"] Nov 04 02:01:23.268159 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.268123 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pw9t7\" (UniqueName: \"kubernetes.io/projected/0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d-kube-api-access-pw9t7\") pod \"dns-operator-controller-manager-8557cbdb84-nfhp7\" (UID: \"0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:23.331417 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.331352 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pw9t7\" (UniqueName: \"kubernetes.io/projected/0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d-kube-api-access-pw9t7\") pod \"dns-operator-controller-manager-8557cbdb84-nfhp7\" (UID: \"0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d\") " pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:23.385442 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.385406 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:23.768451 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.768412 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7"] Nov 04 02:01:23.788741 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:23.788712 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ccae54d_06a6_4d0c_8dd1_55fdfcf7cb1d.slice/crio-1ecf528174b66d34561e07719e7133a3fbc8d8659efa52f859c0158ed92d9bba WatchSource:0}: Error finding container 1ecf528174b66d34561e07719e7133a3fbc8d8659efa52f859c0158ed92d9bba: Status 404 returned error can't find the container with id 1ecf528174b66d34561e07719e7133a3fbc8d8659efa52f859c0158ed92d9bba Nov 04 02:01:23.790918 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:23.790903 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:01:24.745462 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:24.745426 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" event={"ID":"0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d","Type":"ContainerStarted","Data":"1ecf528174b66d34561e07719e7133a3fbc8d8659efa52f859c0158ed92d9bba"} Nov 04 02:01:32.776855 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:32.776819 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" event={"ID":"0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d","Type":"ContainerStarted","Data":"c0783012bc6bbd70e5e7f07a83e32653408d84a218b630f0f5d93eeb21e4532a"} Nov 04 02:01:32.777251 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:32.776912 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:32.896577 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:32.896246 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" podStartSLOduration=1.088114916 podStartE2EDuration="9.896228562s" totalImagesPullingTime="8.808113646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:32.895675202 +0000 UTC m=+325.389000343" watchObservedRunningTime="2026-11-04 02:01:32.896228562 +0000 UTC m=+325.389553703" Nov 04 02:01:34.501662 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:01:34.501626 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:01:35.176265 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.176234 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr"] Nov 04 02:01:35.179207 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.179191 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.247221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.247191 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr"] Nov 04 02:01:35.266404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.266376 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a3053351-606b-4a29-ae0f-34205a984afb-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.266558 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.266419 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3053351-606b-4a29-ae0f-34205a984afb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.266558 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.266442 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67mgv\" (UniqueName: \"kubernetes.io/projected/a3053351-606b-4a29-ae0f-34205a984afb-kube-api-access-67mgv\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.293845 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.293821 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Nov 04 02:01:35.294581 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.294546 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Nov 04 02:01:35.294581 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.294547 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-qh6x5\"" Nov 04 02:01:35.366874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.366849 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a3053351-606b-4a29-ae0f-34205a984afb-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.367001 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.366888 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3053351-606b-4a29-ae0f-34205a984afb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.367001 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.366907 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67mgv\" (UniqueName: \"kubernetes.io/projected/a3053351-606b-4a29-ae0f-34205a984afb-kube-api-access-67mgv\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.367572 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.367502 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/a3053351-606b-4a29-ae0f-34205a984afb-nginx-conf\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.369344 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.369326 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3053351-606b-4a29-ae0f-34205a984afb-plugin-serving-cert\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.430728 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.430671 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67mgv\" (UniqueName: \"kubernetes.io/projected/a3053351-606b-4a29-ae0f-34205a984afb-kube-api-access-67mgv\") pod \"kuadrant-console-plugin-66c4fb575d-hf7tr\" (UID: \"a3053351-606b-4a29-ae0f-34205a984afb\") " pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.490016 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.489992 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" Nov 04 02:01:35.862188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:35.862151 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr"] Nov 04 02:01:35.865142 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:35.865117 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3053351_606b_4a29_ae0f_34205a984afb.slice/crio-284e07c87691813ad1d23b8b53a98acda5eecc2abd4601a05fdeec1e523db01c WatchSource:0}: Error finding container 284e07c87691813ad1d23b8b53a98acda5eecc2abd4601a05fdeec1e523db01c: Status 404 returned error can't find the container with id 284e07c87691813ad1d23b8b53a98acda5eecc2abd4601a05fdeec1e523db01c Nov 04 02:01:36.793633 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:36.793597 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" event={"ID":"a3053351-606b-4a29-ae0f-34205a984afb","Type":"ContainerStarted","Data":"284e07c87691813ad1d23b8b53a98acda5eecc2abd4601a05fdeec1e523db01c"} Nov 04 02:01:36.999378 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:36.999338 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:37.039865 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.039832 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.060386 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.060359 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:37.081658 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.081631 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xshkp\" (UniqueName: \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.081776 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.081675 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.108999 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.108973 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Nov 04 02:01:37.182409 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.182383 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.182574 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.182448 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xshkp\" (UniqueName: \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.183071 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.183049 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.243687 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.243653 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xshkp\" (UniqueName: \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\") pod \"limitador-limitador-74dfcfb894-ckrpb\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.250074 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.250050 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:37.363525 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.363430 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:37.590538 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.590502 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj"] Nov 04 02:01:37.605248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.605224 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.677080 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.677002 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj"] Nov 04 02:01:37.677972 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.677946 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Nov 04 02:01:37.678128 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.678082 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Nov 04 02:01:37.688311 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.688285 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cw7hp\" (UniqueName: \"kubernetes.io/projected/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-kube-api-access-cw7hp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.688457 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.688353 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.688544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.688447 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.704276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.704253 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Nov 04 02:01:37.704452 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.704439 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Nov 04 02:01:37.704792 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.704768 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-krskw\"" Nov 04 02:01:37.790098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.789139 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cw7hp\" (UniqueName: \"kubernetes.io/projected/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-kube-api-access-cw7hp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.790098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.789193 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.790098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.789228 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.798501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.792358 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tmp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.798501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.792898 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-tls-certs\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.798501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.794514 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:37.800755 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:37.800531 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdc0586_e5b6_41ee_b586_55a1b9561afc.slice/crio-fb503a62984fd328f65602d89fe4c08233d7d0ccf20a5b1813da82bfb3cea695 WatchSource:0}: Error finding container fb503a62984fd328f65602d89fe4c08233d7d0ccf20a5b1813da82bfb3cea695: Status 404 returned error can't find the container with id fb503a62984fd328f65602d89fe4c08233d7d0ccf20a5b1813da82bfb3cea695 Nov 04 02:01:37.851649 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.851617 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cw7hp\" (UniqueName: \"kubernetes.io/projected/8f4bbf7f-ffc0-4baf-adce-2c77363ba644-kube-api-access-cw7hp\") pod \"kube-auth-proxy-77fbbd9bf8-5zczj\" (UID: \"8f4bbf7f-ffc0-4baf-adce-2c77363ba644\") " pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:37.917026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:37.916996 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" Nov 04 02:01:38.306783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:38.306755 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj"] Nov 04 02:01:38.309948 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:38.309908 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f4bbf7f_ffc0_4baf_adce_2c77363ba644.slice/crio-95f76a11ac99072db1301ce2a1a19651c6ce95dcfc6bf3641c9a5d36ced93f5d WatchSource:0}: Error finding container 95f76a11ac99072db1301ce2a1a19651c6ce95dcfc6bf3641c9a5d36ced93f5d: Status 404 returned error can't find the container with id 95f76a11ac99072db1301ce2a1a19651c6ce95dcfc6bf3641c9a5d36ced93f5d Nov 04 02:01:38.808872 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:38.808841 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" event={"ID":"4bdc0586-e5b6-41ee-b586-55a1b9561afc","Type":"ContainerStarted","Data":"fb503a62984fd328f65602d89fe4c08233d7d0ccf20a5b1813da82bfb3cea695"} Nov 04 02:01:38.810017 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:38.809989 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" event={"ID":"8f4bbf7f-ffc0-4baf-adce-2c77363ba644","Type":"ContainerStarted","Data":"95f76a11ac99072db1301ce2a1a19651c6ce95dcfc6bf3641c9a5d36ced93f5d"} Nov 04 02:01:41.025089 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.025042 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck"] Nov 04 02:01:41.027393 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.027375 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.080674 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.080641 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"jobset-manager-config\"" Nov 04 02:01:41.080674 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.080644 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"openshift-service-ca.crt\"" Nov 04 02:01:41.119207 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.119164 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7bq5f\" (UniqueName: \"kubernetes.io/projected/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-kube-api-access-7bq5f\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.119389 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.119230 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-metrics-certs\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.119389 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.119286 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-cert\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.119389 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.119317 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-manager-config\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.130341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.130269 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-jobset-operator\"/\"kube-root-ca.crt\"" Nov 04 02:01:41.130534 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.130420 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"metrics-server-cert\"" Nov 04 02:01:41.131746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.131463 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck"] Nov 04 02:01:41.131746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.131641 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"jobset-controller-manager-dockercfg-rcg5g\"" Nov 04 02:01:41.131746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.131721 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-jobset-operator\"/\"webhook-server-cert\"" Nov 04 02:01:41.220781 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.220745 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-cert\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.220972 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.220808 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-manager-config\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.220972 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.220862 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7bq5f\" (UniqueName: \"kubernetes.io/projected/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-kube-api-access-7bq5f\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.220972 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.220894 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-metrics-certs\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.222145 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.222093 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-manager-config\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.224047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.224013 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-metrics-certs\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.224384 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.224363 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-cert\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.282647 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.282584 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7bq5f\" (UniqueName: \"kubernetes.io/projected/7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e-kube-api-access-7bq5f\") pod \"jobset-controller-manager-f46d769d6-8vbck\" (UID: \"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e\") " pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.340872 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.340841 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:41.719162 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.719090 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck"] Nov 04 02:01:41.728344 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:01:41.728315 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f765df6_d37f_4c5a_a4ef_22eb1f29aa7e.slice/crio-24d0659d78dacafe47822301a471244241623b8654724f70c4f505290f6885a5 WatchSource:0}: Error finding container 24d0659d78dacafe47822301a471244241623b8654724f70c4f505290f6885a5: Status 404 returned error can't find the container with id 24d0659d78dacafe47822301a471244241623b8654724f70c4f505290f6885a5 Nov 04 02:01:41.826198 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.826158 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" event={"ID":"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e","Type":"ContainerStarted","Data":"24d0659d78dacafe47822301a471244241623b8654724f70c4f505290f6885a5"} Nov 04 02:01:41.827951 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:41.827917 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" event={"ID":"8f4bbf7f-ffc0-4baf-adce-2c77363ba644","Type":"ContainerStarted","Data":"ac39bb040c544659715e70cf987f52b8dc8390ffa97572805c318aad05e2ef8e"} Nov 04 02:01:43.783392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:43.783315 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-8557cbdb84-nfhp7" Nov 04 02:01:43.838593 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:43.838557 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" event={"ID":"a3053351-606b-4a29-ae0f-34205a984afb","Type":"ContainerStarted","Data":"849991be7d86c9ee64fa09945ae78c1268eb0ffef6ed79f8f87d7d12bf0ee88e"} Nov 04 02:01:43.901882 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:43.901817 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-77fbbd9bf8-5zczj" podStartSLOduration=3.954733213 podStartE2EDuration="6.901800744s" totalImagesPullingTime="2.947067531s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:41.947722721 +0000 UTC m=+334.441047864" watchObservedRunningTime="2026-11-04 02:01:43.901800744 +0000 UTC m=+336.395125886" Nov 04 02:01:44.016086 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:44.015999 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66c4fb575d-hf7tr" podStartSLOduration=1.3623195479999999 podStartE2EDuration="9.015977795s" totalImagesPullingTime="7.653658247s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:44.015248877 +0000 UTC m=+336.508574021" watchObservedRunningTime="2026-11-04 02:01:44.015977795 +0000 UTC m=+336.509302944" Nov 04 02:01:46.853054 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:46.853019 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" event={"ID":"7f765df6-d37f-4c5a-a4ef-22eb1f29aa7e","Type":"ContainerStarted","Data":"77fa7fd293d4a66b575d84d485ac773c78b4b9354648d24525859000e60c8f9c"} Nov 04 02:01:46.853442 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:46.853130 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:46.970403 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:46.970351 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" podStartSLOduration=2.748887671 podStartE2EDuration="6.970337509s" totalImagesPullingTime="4.221449838s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:46.968858742 +0000 UTC m=+339.462183883" watchObservedRunningTime="2026-11-04 02:01:46.970337509 +0000 UTC m=+339.463662650" Nov 04 02:01:52.877861 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:52.877760 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" event={"ID":"4bdc0586-e5b6-41ee-b586-55a1b9561afc","Type":"ContainerStarted","Data":"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f"} Nov 04 02:01:52.878237 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:52.877860 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:52.995364 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:52.995307 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" podStartSLOduration=2.2967050159999998 podStartE2EDuration="16.99529188s" totalImagesPullingTime="14.698586864s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:01:36 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:01:52.993955302 +0000 UTC m=+345.487280467" watchObservedRunningTime="2026-11-04 02:01:52.99529188 +0000 UTC m=+345.488617021" Nov 04 02:01:57.564859 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:57.564821 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:57.565443 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:57.565002 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" podUID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" containerName="limitador" containerID="cri-o://b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f" gracePeriod=30 Nov 04 02:01:57.565832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:57.565745 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:57.862516 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:57.862414 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-jobset-operator/jobset-controller-manager-f46d769d6-8vbck" Nov 04 02:01:58.118539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.118463 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:58.261951 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.261918 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\" (UniqueName: \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\") pod \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " Nov 04 02:01:58.262121 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.261966 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\" (UniqueName: \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\") pod \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\" (UID: \"4bdc0586-e5b6-41ee-b586-55a1b9561afc\") " Nov 04 02:01:58.262331 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.262308 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file" pod "4bdc0586-e5b6-41ee-b586-55a1b9561afc" (UID: "4bdc0586-e5b6-41ee-b586-55a1b9561afc"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:01:58.264337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.264309 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp" pod "4bdc0586-e5b6-41ee-b586-55a1b9561afc" (UID: "4bdc0586-e5b6-41ee-b586-55a1b9561afc"). InnerVolumeSpecName "kube-api-access-xshkp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:01:58.362621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.362582 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xshkp\" (UniqueName: \"kubernetes.io/projected/4bdc0586-e5b6-41ee-b586-55a1b9561afc-kube-api-access-xshkp\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:01:58.362621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.362614 5029 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/4bdc0586-e5b6-41ee-b586-55a1b9561afc-config-file\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:01:58.902836 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.902795 5029 generic.go:356] "Generic (PLEG): container finished" podID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" containerID="b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f" exitCode=0 Nov 04 02:01:58.903247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.902848 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" Nov 04 02:01:58.903247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.902889 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" event={"ID":"4bdc0586-e5b6-41ee-b586-55a1b9561afc","Type":"ContainerDied","Data":"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f"} Nov 04 02:01:58.903247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.902946 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-74dfcfb894-ckrpb" event={"ID":"4bdc0586-e5b6-41ee-b586-55a1b9561afc","Type":"ContainerDied","Data":"fb503a62984fd328f65602d89fe4c08233d7d0ccf20a5b1813da82bfb3cea695"} Nov 04 02:01:58.903247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.902971 5029 scope.go:122] "RemoveContainer" containerID="b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f" Nov 04 02:01:58.911760 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.911741 5029 scope.go:122] "RemoveContainer" containerID="b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f" Nov 04 02:01:58.912055 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:01:58.912035 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f\": container with ID starting with b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f not found: ID does not exist" containerID="b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f" Nov 04 02:01:58.912093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:58.912065 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f"} err="failed to get container status \"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f\": rpc error: code = NotFound desc = could not find container \"b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f\": container with ID starting with b7808e4a317150a8a412e7f3a395d909722a64fe9064b4c68ef6b97829326a6f not found: ID does not exist" Nov 04 02:01:59.065134 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:59.065092 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:59.114748 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:59.114717 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-74dfcfb894-ckrpb"] Nov 04 02:01:59.619083 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:01:59.619050 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" path="/var/lib/kubelet/pods/4bdc0586-e5b6-41ee-b586-55a1b9561afc/volumes" Nov 04 02:02:04.502382 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:02:04.502345 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:02:34.502428 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:02:34.502388 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:03:04.502310 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:03:04.502269 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:03:15.360376 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.360340 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:03:15.360859 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.360696 5029 cpu_manager.go:414] "containerMap: removing container" podUID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" containerName="limitador" Nov 04 02:03:15.360859 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.360708 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" containerName="limitador" Nov 04 02:03:15.360859 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.360776 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="4bdc0586-e5b6-41ee-b586-55a1b9561afc" containerName="limitador" Nov 04 02:03:15.363775 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.363754 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.404627 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.404587 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.404627 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.404626 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxsx6\" (UniqueName: \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.465966 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.465928 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Nov 04 02:03:15.465966 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.465952 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-lnk9f\"" Nov 04 02:03:15.466204 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.465984 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Nov 04 02:03:15.466204 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.466065 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Nov 04 02:03:15.467062 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.467041 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:03:15.505896 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.505859 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.506077 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.505903 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sxsx6\" (UniqueName: \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.508484 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.508452 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.567177 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.567144 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxsx6\" (UniqueName: \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\") pod \"llmisvc-controller-manager-869d5dcc5d-q9lfs\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:15.676184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:15.676086 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:16.045323 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:16.045287 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:03:16.048674 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:03:16.048641 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod66b6dc88_2748_4a85_849f_d79923dd7dd6.slice/crio-af252a07ac2af5e48e53cddff73499c55f9a051737d7527b4c6c0a8f02adac0f WatchSource:0}: Error finding container af252a07ac2af5e48e53cddff73499c55f9a051737d7527b4c6c0a8f02adac0f: Status 404 returned error can't find the container with id af252a07ac2af5e48e53cddff73499c55f9a051737d7527b4c6c0a8f02adac0f Nov 04 02:03:16.211163 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:16.211124 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerStarted","Data":"af252a07ac2af5e48e53cddff73499c55f9a051737d7527b4c6c0a8f02adac0f"} Nov 04 02:03:23.241169 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:23.241116 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerStarted","Data":"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3"} Nov 04 02:03:23.241631 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:23.241196 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:23.358261 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:23.358157 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" podStartSLOduration=1.408032441 podStartE2EDuration="8.35814262s" totalImagesPullingTime="6.950110179s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:23.357450598 +0000 UTC m=+435.850775741" watchObservedRunningTime="2026-11-04 02:03:23.35814262 +0000 UTC m=+435.851467760" Nov 04 02:03:34.285029 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:34.284988 5029 generic.go:356] "Generic (PLEG): container finished" podID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerID="cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3" exitCode=1 Nov 04 02:03:34.285511 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:34.285061 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerDied","Data":"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3"} Nov 04 02:03:34.285511 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:34.285413 5029 scope.go:122] "RemoveContainer" containerID="cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3" Nov 04 02:03:34.502674 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:03:34.502633 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:03:35.290913 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:35.290879 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerStarted","Data":"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0"} Nov 04 02:03:35.291344 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:35.291087 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:03:35.972737 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:35.972697 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g"] Nov 04 02:03:35.976024 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:35.976007 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.028792 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.028758 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Nov 04 02:03:36.078145 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.078113 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Nov 04 02:03:36.078145 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.078116 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Nov 04 02:03:36.078397 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.078158 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-hnsgt\"" Nov 04 02:03:36.078397 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.078189 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-metrics-tls\"" Nov 04 02:03:36.079416 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.079396 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g"] Nov 04 02:03:36.079687 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.079666 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90875d56-cf9b-402f-9bd5-fcb993cf91d3-openshift-service-ca\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.079734 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.079698 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-maas-api-tls\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.079734 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.079715 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nkhzh\" (UniqueName: \"kubernetes.io/projected/90875d56-cf9b-402f-9bd5-fcb993cf91d3-kube-api-access-nkhzh\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.079812 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.079745 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-metrics-certs\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.180294 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.180263 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90875d56-cf9b-402f-9bd5-fcb993cf91d3-openshift-service-ca\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.180294 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.180299 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-maas-api-tls\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.180598 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.180319 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nkhzh\" (UniqueName: \"kubernetes.io/projected/90875d56-cf9b-402f-9bd5-fcb993cf91d3-kube-api-access-nkhzh\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.180598 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.180345 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-metrics-certs\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.181128 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.181031 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/90875d56-cf9b-402f-9bd5-fcb993cf91d3-openshift-service-ca\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.183203 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.183176 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-metrics-certs\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.183203 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.183198 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/90875d56-cf9b-402f-9bd5-fcb993cf91d3-maas-api-tls\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.241324 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.241291 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nkhzh\" (UniqueName: \"kubernetes.io/projected/90875d56-cf9b-402f-9bd5-fcb993cf91d3-kube-api-access-nkhzh\") pod \"maas-api-5cf77b77f7-zsw8g\" (UID: \"90875d56-cf9b-402f-9bd5-fcb993cf91d3\") " pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.288300 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.288263 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:36.661676 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:36.661603 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g"] Nov 04 02:03:36.664802 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:03:36.664771 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90875d56_cf9b_402f_9bd5_fcb993cf91d3.slice/crio-82de56abd0725f94a80daa3d2fcc1e6adba457b333bca8d6138c3394d3009a18 WatchSource:0}: Error finding container 82de56abd0725f94a80daa3d2fcc1e6adba457b333bca8d6138c3394d3009a18: Status 404 returned error can't find the container with id 82de56abd0725f94a80daa3d2fcc1e6adba457b333bca8d6138c3394d3009a18 Nov 04 02:03:37.301545 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:37.301496 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" event={"ID":"90875d56-cf9b-402f-9bd5-fcb993cf91d3","Type":"ContainerStarted","Data":"82de56abd0725f94a80daa3d2fcc1e6adba457b333bca8d6138c3394d3009a18"} Nov 04 02:03:41.323760 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:41.323721 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" event={"ID":"90875d56-cf9b-402f-9bd5-fcb993cf91d3","Type":"ContainerStarted","Data":"5a6266a50bd626eaceb425be72645a56b646a9e9518157c539cc22e91eb03901"} Nov 04 02:03:41.324157 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:41.323862 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:03:41.442442 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:41.442386 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" podStartSLOduration=2.165228321 podStartE2EDuration="6.442371252s" totalImagesPullingTime="4.277142931s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:03:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:03:41.441196106 +0000 UTC m=+453.934521247" watchObservedRunningTime="2026-11-04 02:03:41.442371252 +0000 UTC m=+453.935696390" Nov 04 02:03:47.332944 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:03:47.332869 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-5cf77b77f7-zsw8g" Nov 04 02:04:04.502135 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:04:04.502092 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:04:06.296769 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:04:06.296739 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:04:34.501895 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:04:34.501847 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:05:00.973691 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:00.973656 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9"] Nov 04 02:05:00.976679 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:00.976663 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:00.984151 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:00.984119 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rvf4z\" (UniqueName: \"kubernetes.io/projected/8b56c643-0cf2-4293-b1a1-fb9fde73954c-kube-api-access-rvf4z\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:00.984291 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:00.984178 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b56c643-0cf2-4293-b1a1-fb9fde73954c-cert\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.035260 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.035234 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9"] Nov 04 02:05:01.085286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.085256 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rvf4z\" (UniqueName: \"kubernetes.io/projected/8b56c643-0cf2-4293-b1a1-fb9fde73954c-kube-api-access-rvf4z\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.085449 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.085310 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b56c643-0cf2-4293-b1a1-fb9fde73954c-cert\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.087932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.087911 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/8b56c643-0cf2-4293-b1a1-fb9fde73954c-cert\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.146487 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.146440 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rvf4z\" (UniqueName: \"kubernetes.io/projected/8b56c643-0cf2-4293-b1a1-fb9fde73954c-kube-api-access-rvf4z\") pod \"llmisvc-controller-manager-6466c554d8-gkgm9\" (UID: \"8b56c643-0cf2-4293-b1a1-fb9fde73954c\") " pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.290829 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.290797 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:01.661377 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:01.661287 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9"] Nov 04 02:05:01.664235 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:05:01.664209 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b56c643_0cf2_4293_b1a1_fb9fde73954c.slice/crio-06879d05d1622ab5434dfcf89ef9fc9b4cfb3da85178e37e7092a1b766b08c50 WatchSource:0}: Error finding container 06879d05d1622ab5434dfcf89ef9fc9b4cfb3da85178e37e7092a1b766b08c50: Status 404 returned error can't find the container with id 06879d05d1622ab5434dfcf89ef9fc9b4cfb3da85178e37e7092a1b766b08c50 Nov 04 02:05:02.633407 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:02.633375 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" event={"ID":"8b56c643-0cf2-4293-b1a1-fb9fde73954c","Type":"ContainerStarted","Data":"aa5cd7a305e1662d4f8f047cd6f4acd5c299787057b1a5df6c5130ad7c0b949b"} Nov 04 02:05:02.633907 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:02.633421 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" event={"ID":"8b56c643-0cf2-4293-b1a1-fb9fde73954c","Type":"ContainerStarted","Data":"06879d05d1622ab5434dfcf89ef9fc9b4cfb3da85178e37e7092a1b766b08c50"} Nov 04 02:05:02.633907 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:02.633454 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:02.750864 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:02.750823 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" podStartSLOduration=1.993727032 podStartE2EDuration="2.750809837s" totalImagesPullingTime="757.082805ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:05:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:05:02.749614809 +0000 UTC m=+535.242939951" watchObservedRunningTime="2026-11-04 02:05:02.750809837 +0000 UTC m=+535.244134978" Nov 04 02:05:04.502528 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:05:04.502405 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:05:33.640340 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:33.640309 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-6466c554d8-gkgm9" Nov 04 02:05:33.807447 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:33.807413 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:05:33.807730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:33.807678 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" containerID="cri-o://100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0" gracePeriod=30 Nov 04 02:05:34.056716 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.056690 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:05:34.129380 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.129341 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\" (UniqueName: \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\") pod \"66b6dc88-2748-4a85-849f-d79923dd7dd6\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " Nov 04 02:05:34.129580 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.129446 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\" (UniqueName: \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\") pod \"66b6dc88-2748-4a85-849f-d79923dd7dd6\" (UID: \"66b6dc88-2748-4a85-849f-d79923dd7dd6\") " Nov 04 02:05:34.131857 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.131821 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6" pod "66b6dc88-2748-4a85-849f-d79923dd7dd6" (UID: "66b6dc88-2748-4a85-849f-d79923dd7dd6"). InnerVolumeSpecName "kube-api-access-sxsx6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:05:34.131857 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.131837 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert" pod "66b6dc88-2748-4a85-849f-d79923dd7dd6" (UID: "66b6dc88-2748-4a85-849f-d79923dd7dd6"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:05:34.230749 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.230661 5029 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/66b6dc88-2748-4a85-849f-d79923dd7dd6-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:05:34.230749 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.230694 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxsx6\" (UniqueName: \"kubernetes.io/projected/66b6dc88-2748-4a85-849f-d79923dd7dd6-kube-api-access-sxsx6\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:05:34.502898 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:05:34.502863 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:05:34.760079 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.759993 5029 generic.go:356] "Generic (PLEG): container finished" podID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerID="100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0" exitCode=0 Nov 04 02:05:34.760079 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.760055 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerDied","Data":"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0"} Nov 04 02:05:34.760544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.760142 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" event={"ID":"66b6dc88-2748-4a85-849f-d79923dd7dd6","Type":"ContainerDied","Data":"af252a07ac2af5e48e53cddff73499c55f9a051737d7527b4c6c0a8f02adac0f"} Nov 04 02:05:34.760544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.760066 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs" Nov 04 02:05:34.760544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.760175 5029 scope.go:122] "RemoveContainer" containerID="100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0" Nov 04 02:05:34.769309 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.769291 5029 scope.go:122] "RemoveContainer" containerID="cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3" Nov 04 02:05:34.776725 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.776706 5029 scope.go:122] "RemoveContainer" containerID="100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0" Nov 04 02:05:34.776999 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:05:34.776981 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0\": container with ID starting with 100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0 not found: ID does not exist" containerID="100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0" Nov 04 02:05:34.777037 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.777009 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0"} err="failed to get container status \"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0\": rpc error: code = NotFound desc = could not find container \"100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0\": container with ID starting with 100579cc08fcf6e03373181d8ee49c59530ee29fcf8cf3f2b0ffaf263bdb70c0 not found: ID does not exist" Nov 04 02:05:34.777037 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.777027 5029 scope.go:122] "RemoveContainer" containerID="cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3" Nov 04 02:05:34.777262 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:05:34.777243 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3\": container with ID starting with cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3 not found: ID does not exist" containerID="cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3" Nov 04 02:05:34.777298 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.777270 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3"} err="failed to get container status \"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3\": rpc error: code = NotFound desc = could not find container \"cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3\": container with ID starting with cc124961fd092dbfc552c84891d8c3ddf63bebb81e6de874d36285f20ec261f3 not found: ID does not exist" Nov 04 02:05:34.924292 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.924256 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:05:34.974080 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:34.974040 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["opendatahub/llmisvc-controller-manager-869d5dcc5d-q9lfs"] Nov 04 02:05:35.618785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:05:35.618755 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" path="/var/lib/kubelet/pods/66b6dc88-2748-4a85-849f-d79923dd7dd6/volumes" Nov 04 02:06:04.502191 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:06:04.502144 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:06:34.502087 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:06:34.502042 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:07:04.502038 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:07:04.501958 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:07:22.352863 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.352831 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf"] Nov 04 02:07:22.353326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.353254 5029 cpu_manager.go:414] "containerMap: removing container" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:22.353326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.353268 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:22.353397 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.353364 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:22.356453 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.356436 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.408783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.408759 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-wgllv\"" Nov 04 02:07:22.453826 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.453795 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.453969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.453857 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qdzm\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-kube-api-access-2qdzm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.453969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.453932 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.453976 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.453999 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454134 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.454065 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454168 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.454132 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454201 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.454166 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.454201 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.454184 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.458369 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.458355 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Nov 04 02:07:22.459750 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.459729 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf"] Nov 04 02:07:22.554830 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.554798 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qdzm\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-kube-api-access-2qdzm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.554830 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.554831 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555068 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.554851 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555068 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.554868 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555068 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555021 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555079 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555127 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555157 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555193 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555274 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555274 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555398 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555615 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555437 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.555847 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.555825 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.557597 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.557576 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.557879 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.557864 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.618622 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.618539 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.618786 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.618642 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qdzm\" (UniqueName: \"kubernetes.io/projected/fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62-kube-api-access-2qdzm\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf\" (UID: \"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:22.669142 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:22.669089 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:23.095984 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:23.095959 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf"] Nov 04 02:07:23.098465 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:07:23.098435 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb391b4_37c6_48a7_9fa2_a7f67ed4ee62.slice/crio-14adb3023be81964ca86bb7986a4afa80a7fffd970b033065acb71624ab84216 WatchSource:0}: Error finding container 14adb3023be81964ca86bb7986a4afa80a7fffd970b033065acb71624ab84216: Status 404 returned error can't find the container with id 14adb3023be81964ca86bb7986a4afa80a7fffd970b033065acb71624ab84216 Nov 04 02:07:23.100533 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:23.100515 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:07:23.188341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:23.188304 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" event={"ID":"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62","Type":"ContainerStarted","Data":"14adb3023be81964ca86bb7986a4afa80a7fffd970b033065acb71624ab84216"} Nov 04 02:07:26.548240 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:26.548207 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:26.548657 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:26.548274 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:26.548657 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:26.548301 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:27.208212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:27.208178 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" event={"ID":"fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62","Type":"ContainerStarted","Data":"d95a03c0c2e05b5bd7d1aab5f39af424ae6b6e0078800bafc3277b5ed07f9bce"} Nov 04 02:07:27.431213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:27.431148 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podStartSLOduration=1.984487152 podStartE2EDuration="5.431128708s" totalImagesPullingTime="3.446641556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:27.380741562 +0000 UTC m=+679.874066766" watchObservedRunningTime="2026-11-04 02:07:27.431128708 +0000 UTC m=+679.924453850" Nov 04 02:07:27.670278 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:27.670239 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:27.671815 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:27.671786 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:27.671948 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:27.671836 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:28.670085 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:28.670047 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:28.670294 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:28.670109 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:29.669679 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:29.669641 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:29.669874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:29.669699 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:30.670490 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:30.670445 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:30.670978 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:30.670524 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:31.669728 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:31.669692 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:31.669896 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:31.669747 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:32.670223 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:32.670186 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:32.670634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:32.670252 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:33.669564 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:33.669519 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" start-of-body= Nov 04 02:07:33.669739 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:33.669598 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" podUID="fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.26:15021/healthz/ready\": dial tcp 10.133.0.26:15021: connect: connection refused" Nov 04 02:07:34.504316 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:07:34.504277 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:07:35.072519 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:35.072448 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:35.073304 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:35.073241 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:35.073932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:35.073911 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf" Nov 04 02:07:37.332214 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.332180 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:07:37.332730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.332678 5029 cpu_manager.go:414] "containerMap: removing container" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:37.332730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.332696 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:37.332845 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.332788 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="66b6dc88-2748-4a85-849f-d79923dd7dd6" containerName="manager" Nov 04 02:07:37.335919 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.335897 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506362 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506334 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506379 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506447 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506546 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bc9j2\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506589 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506614 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506639 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506755 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506674 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.506755 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.506694 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.548585 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.548545 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:07:37.607815 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607735 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.607815 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607780 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bc9j2\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.607815 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607811 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607832 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607855 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607873 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607890 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.607921 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608328 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608132 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608328 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608195 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608438 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608325 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608438 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608347 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608438 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608428 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.608736 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.608713 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.610333 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.610311 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.610649 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.610631 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.669184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.669156 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.669512 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.669463 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bc9j2\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-jchrk\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:37.947406 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:37.947331 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:38.371271 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:38.371240 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:07:38.374617 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:07:38.374555 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod310608eb_1844_4900_ace6_4ccfd0ee7d86.slice/crio-0e101b963451ec276ccef3b3dfbfb29cec81f8bc52138b265bc06cfcde52584b WatchSource:0}: Error finding container 0e101b963451ec276ccef3b3dfbfb29cec81f8bc52138b265bc06cfcde52584b: Status 404 returned error can't find the container with id 0e101b963451ec276ccef3b3dfbfb29cec81f8bc52138b265bc06cfcde52584b Nov 04 02:07:38.376631 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:38.376598 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:38.376706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:38.376681 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:38.376743 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:38.376711 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:07:39.271079 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.271041 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" event={"ID":"310608eb-1844-4900-ace6-4ccfd0ee7d86","Type":"ContainerStarted","Data":"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4"} Nov 04 02:07:39.271267 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.271138 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" event={"ID":"310608eb-1844-4900-ace6-4ccfd0ee7d86","Type":"ContainerStarted","Data":"0e101b963451ec276ccef3b3dfbfb29cec81f8bc52138b265bc06cfcde52584b"} Nov 04 02:07:39.493014 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.492951 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podStartSLOduration=2.492934505 podStartE2EDuration="2.492934505s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:07:37 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:07:39.443091005 +0000 UTC m=+691.936416149" watchObservedRunningTime="2026-11-04 02:07:39.492934505 +0000 UTC m=+691.986259643" Nov 04 02:07:39.956370 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.955857 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:39.956370 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.955981 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:39.956370 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:39.956036 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:40.954296 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:40.953838 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:40.954296 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:40.953899 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:41.951495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:41.950861 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:41.951495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:41.950926 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:42.949501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:42.948823 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:42.949501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:42.948903 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:43.949871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:43.949830 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:43.950575 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:43.950541 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:44.948673 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:44.948641 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:44.948842 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:44.948695 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:45.947932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:45.947894 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" start-of-body= Nov 04 02:07:45.948318 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:45.947949 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.27:15021/healthz/ready\": dial tcp 10.133.0.27:15021: connect: connection refused" Nov 04 02:07:46.951288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:46.951257 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:46.951678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:46.951517 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:07:46.952272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:07:46.952254 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:08:04.501802 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:08:04.501763 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:08:34.501833 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:08:34.501789 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:09:04.502288 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:04.502252 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:09:06.754644 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.754597 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:09:06.757995 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.757976 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.810421 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.810379 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w0-24bea9-serving-cert\"" Nov 04 02:09:06.861379 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.861344 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:09:06.872986 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.872955 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.873144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.873031 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r6txq\" (UniqueName: \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.873286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.873147 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.873286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.873227 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.974369 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.974340 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.974560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.974390 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r6txq\" (UniqueName: \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.974560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.974410 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.974560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.974458 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.974700 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:06.974581 5029 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-serving-cert: secret "maas-api-e2e-worker-w0-24bea9-serving-cert" not found Nov 04 02:09:06.974700 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:06.974647 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls podName:07aa0d55-0d45-4a69-ab12-105745a1e275 nodeName:}" failed. No retries permitted until 2026-11-04 02:09:07.474632387 +0000 UTC m=+779.967957505 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls") pod "maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" (UID: "07aa0d55-0d45-4a69-ab12-105745a1e275") : secret "maas-api-e2e-worker-w0-24bea9-serving-cert" not found Nov 04 02:09:06.975080 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.975051 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:06.976857 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:06.976840 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:07.035680 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:07.035616 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r6txq\" (UniqueName: \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:07.479162 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:07.479083 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:07.481785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:07.481757 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") pod \"maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:07.670213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:07.670183 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:08.043583 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:08.043545 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:09:08.046426 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:09:08.046397 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07aa0d55_0d45_4a69_ab12_105745a1e275.slice/crio-7873e72b044d17d60895cbdf74e884edddd99e3980e2ea13f0c964a0725fb042 WatchSource:0}: Error finding container 7873e72b044d17d60895cbdf74e884edddd99e3980e2ea13f0c964a0725fb042: Status 404 returned error can't find the container with id 7873e72b044d17d60895cbdf74e884edddd99e3980e2ea13f0c964a0725fb042 Nov 04 02:09:08.639093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:08.639058 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" event={"ID":"07aa0d55-0d45-4a69-ab12-105745a1e275","Type":"ContainerStarted","Data":"7873e72b044d17d60895cbdf74e884edddd99e3980e2ea13f0c964a0725fb042"} Nov 04 02:09:09.644269 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:09.644235 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" event={"ID":"07aa0d55-0d45-4a69-ab12-105745a1e275","Type":"ContainerStarted","Data":"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742"} Nov 04 02:09:09.644675 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:09.644296 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:09.761778 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:09.761727 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" podStartSLOduration=2.69506867 podStartE2EDuration="3.761713311s" totalImagesPullingTime="1.066644641s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:09.76072673 +0000 UTC m=+782.254051871" watchObservedRunningTime="2026-11-04 02:09:09.761713311 +0000 UTC m=+782.255038452" Nov 04 02:09:11.676790 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.676756 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:09:11.679775 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.679760 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.733962 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.733061 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-worker-w2-29e7e8-serving-cert\"" Nov 04 02:09:11.782980 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.782954 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:09:11.815375 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.815353 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zptd5\" (UniqueName: \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.815531 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.815398 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.815531 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.815518 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.815649 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.815571 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.916303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.916269 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.916506 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.916328 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.916506 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.916365 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.916506 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:11.916431 5029 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-serving-cert: secret "maas-api-e2e-worker-w2-29e7e8-serving-cert" not found Nov 04 02:09:11.916676 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.916431 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zptd5\" (UniqueName: \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.916676 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:11.916541 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls podName:4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0 nodeName:}" failed. No retries permitted until 2026-11-04 02:09:12.416520995 +0000 UTC m=+784.909846134 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls") pod "maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" (UID: "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0") : secret "maas-api-e2e-worker-w2-29e7e8-serving-cert" not found Nov 04 02:09:11.917087 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.917066 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.919074 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.919054 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:11.979566 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:11.979487 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zptd5\" (UniqueName: \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:12.421297 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:12.421264 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:12.424414 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:12.424384 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") pod \"maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:12.592631 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:12.592595 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:12.963665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:12.963575 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:09:12.965958 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:09:12.965931 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4667b36a_071c_4d7b_bc3c_b8a31ae2ebb0.slice/crio-11e128c56ca4c72f6bbc626bdd77d8b98ee1b586dc14cdf31700fd462354ed4a WatchSource:0}: Error finding container 11e128c56ca4c72f6bbc626bdd77d8b98ee1b586dc14cdf31700fd462354ed4a: Status 404 returned error can't find the container with id 11e128c56ca4c72f6bbc626bdd77d8b98ee1b586dc14cdf31700fd462354ed4a Nov 04 02:09:13.663135 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:13.663101 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" event={"ID":"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0","Type":"ContainerStarted","Data":"11e128c56ca4c72f6bbc626bdd77d8b98ee1b586dc14cdf31700fd462354ed4a"} Nov 04 02:09:14.668261 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:14.668223 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" event={"ID":"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0","Type":"ContainerStarted","Data":"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762"} Nov 04 02:09:14.668673 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:14.668333 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:14.786560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:14.786513 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" podStartSLOduration=2.938534012 podStartE2EDuration="3.786494683s" totalImagesPullingTime="847.960671ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:14.785487418 +0000 UTC m=+787.278812578" watchObservedRunningTime="2026-11-04 02:09:14.786494683 +0000 UTC m=+787.279819824" Nov 04 02:09:15.653374 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:15.653350 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:09:20.678053 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:20.678022 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:09:34.501388 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:09:34.501351 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:09:38.851073 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.851037 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:09:38.854347 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.854330 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.906737 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.906712 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3bf869\"/\"kube-root-ca.crt\"" Nov 04 02:09:38.928878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.928849 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w3-3bf869\"/\"openshift-service-ca.crt\"" Nov 04 02:09:38.956521 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.956464 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3bf869\"/\"default-dockercfg-q69bt\"" Nov 04 02:09:38.956699 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.956599 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3bf869\"/\"simulator-w3-3bf869-kserve-self-signed-certs\"" Nov 04 02:09:38.957855 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.957835 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:09:38.960967 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.960951 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.961020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.960992 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.961075 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.961020 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.961155 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.961135 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.961193 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.961169 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:38.961193 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:38.961187 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7sx9z\" (UniqueName: \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061801 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061767 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061990 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061827 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061990 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061869 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061990 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061917 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061990 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061950 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.061990 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.061972 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7sx9z\" (UniqueName: \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.062242 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.062205 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.062297 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.062267 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.062354 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.062292 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.064345 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.064322 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.064632 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.064612 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.125263 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.125191 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7sx9z\" (UniqueName: \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\") pod \"simulator-w3-3bf869-kserve-54bd6944b8-7xfbx\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.166507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.166463 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:39.542881 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.542787 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:09:39.545239 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:09:39.545211 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddc4659f1_1866_4e10_84dc_e6942a2142f5.slice/crio-34ca4d484a7297f1f2e3061be0e7f8f945de2c07a74ea3cfd7464bf084143c2c WatchSource:0}: Error finding container 34ca4d484a7297f1f2e3061be0e7f8f945de2c07a74ea3cfd7464bf084143c2c: Status 404 returned error can't find the container with id 34ca4d484a7297f1f2e3061be0e7f8f945de2c07a74ea3cfd7464bf084143c2c Nov 04 02:09:39.785446 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:39.785410 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" event={"ID":"dc4659f1-1866-4e10-84dc-e6942a2142f5","Type":"ContainerStarted","Data":"34ca4d484a7297f1f2e3061be0e7f8f945de2c07a74ea3cfd7464bf084143c2c"} Nov 04 02:09:41.802271 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:41.802228 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" event={"ID":"dc4659f1-1866-4e10-84dc-e6942a2142f5","Type":"ContainerStarted","Data":"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc"} Nov 04 02:09:41.920768 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:41.920720 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" podStartSLOduration=2.421453155 podStartE2EDuration="3.920707293s" totalImagesPullingTime="1.499254138s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:09:38 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:09:41.919218338 +0000 UTC m=+814.412543505" watchObservedRunningTime="2026-11-04 02:09:41.920707293 +0000 UTC m=+814.414032434" Nov 04 02:09:42.167318 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:42.167238 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:49.167448 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:49.167368 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:49.179885 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:49.179865 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:09:49.846316 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:09:49.846285 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:10:04.502771 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:10:04.502668 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:10:07.840085 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.840053 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:10:07.844252 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.844227 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923023 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.922994 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923023 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.923025 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923200 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.923061 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-db2vk\" (UniqueName: \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923200 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.923179 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923279 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.923213 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.923320 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.923297 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:07.946156 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.946131 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"kube-root-ca.crt\"" Nov 04 02:10:07.946156 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.946139 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"simulator-w0-24bea9-kserve-self-signed-certs\"" Nov 04 02:10:07.946288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.946137 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"openshift-service-ca.crt\"" Nov 04 02:10:07.946288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.946170 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w0-24bea9\"/\"default-dockercfg-cz6zr\"" Nov 04 02:10:07.947696 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:07.947676 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:10:08.024451 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024422 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024457 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024529 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024569 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024588 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024634 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024626 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-db2vk\" (UniqueName: \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024908 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024885 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.024979 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.024922 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.025039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.025007 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.026912 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.026896 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.027253 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.027235 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.085881 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.085851 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-db2vk\" (UniqueName: \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\") pod \"simulator-w0-24bea9-kserve-5669f5fd56-szbpd\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.155626 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.155532 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:08.539529 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.539495 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:10:08.542374 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:10:08.542347 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod29a11f02_1e58_407e_a7b5_5dbbdd48d4a3.slice/crio-925290135b9790b5c61ab71e4fe9269fd2abc49adb5be48246a10ec763ab70c8 WatchSource:0}: Error finding container 925290135b9790b5c61ab71e4fe9269fd2abc49adb5be48246a10ec763ab70c8: Status 404 returned error can't find the container with id 925290135b9790b5c61ab71e4fe9269fd2abc49adb5be48246a10ec763ab70c8 Nov 04 02:10:08.920048 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.919959 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" event={"ID":"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3","Type":"ContainerStarted","Data":"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae"} Nov 04 02:10:08.920048 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:08.919999 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" event={"ID":"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3","Type":"ContainerStarted","Data":"925290135b9790b5c61ab71e4fe9269fd2abc49adb5be48246a10ec763ab70c8"} Nov 04 02:10:09.037970 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:09.037925 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" podStartSLOduration=2.037911548 podStartE2EDuration="2.037911548s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:09.03676327 +0000 UTC m=+841.530088411" watchObservedRunningTime="2026-11-04 02:10:09.037911548 +0000 UTC m=+841.531236689" Nov 04 02:10:09.155924 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:09.155889 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:18.156126 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:18.156088 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:18.168914 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:18.168888 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:18.975070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:18.975041 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:10:19.666140 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.666104 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:10:19.671833 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.671817 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730502 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730450 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730667 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730576 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730714 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730673 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730714 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730696 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t4mpt\" (UniqueName: \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730734 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.730803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.730790 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.775621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.775591 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"openshift-service-ca.crt\"" Nov 04 02:10:19.775753 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.775631 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"kube-root-ca.crt\"" Nov 04 02:10:19.775753 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.775721 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"default-dockercfg-xxf9g\"" Nov 04 02:10:19.775870 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.775859 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"simulator-w2-29e7e8-kserve-self-signed-certs\"" Nov 04 02:10:19.776987 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.776969 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:10:19.832016 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.831989 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832031 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832080 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832097 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t4mpt\" (UniqueName: \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832127 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832262 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832571 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832544 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832622 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832605 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.832774 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.832757 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.834582 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.834558 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.835245 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.835224 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.893436 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.893362 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t4mpt\" (UniqueName: \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\") pod \"simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:19.983730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:19.983699 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:20.368287 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:20.368265 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:10:20.370985 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:10:20.370958 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9113fc57_3a0b_449c_831e_207de47de756.slice/crio-2a89d56f6f413d69d5c399b6bbcb609e7a2e4eba97bd383685dfaa6805b5a4fd WatchSource:0}: Error finding container 2a89d56f6f413d69d5c399b6bbcb609e7a2e4eba97bd383685dfaa6805b5a4fd: Status 404 returned error can't find the container with id 2a89d56f6f413d69d5c399b6bbcb609e7a2e4eba97bd383685dfaa6805b5a4fd Nov 04 02:10:20.975061 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:20.975024 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" event={"ID":"9113fc57-3a0b-449c-831e-207de47de756","Type":"ContainerStarted","Data":"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b"} Nov 04 02:10:20.975574 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:20.975069 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" event={"ID":"9113fc57-3a0b-449c-831e-207de47de756","Type":"ContainerStarted","Data":"2a89d56f6f413d69d5c399b6bbcb609e7a2e4eba97bd383685dfaa6805b5a4fd"} Nov 04 02:10:20.984308 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:20.984285 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:21.092199 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:21.092153 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" podStartSLOduration=2.092141371 podStartE2EDuration="2.092141371s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:21.091406955 +0000 UTC m=+853.584732111" watchObservedRunningTime="2026-11-04 02:10:21.092141371 +0000 UTC m=+853.585466511" Nov 04 02:10:29.984645 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:29.984608 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:29.997105 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:29.997075 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:30.024043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:30.024012 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:10:34.502173 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:10:34.502134 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:10:41.670658 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.670621 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:10:41.674442 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.674420 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723264 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723232 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723276 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723509 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723441 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723571 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723536 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723608 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723579 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jt47f\" (UniqueName: \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.723726 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.723709 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.778192 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.778165 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w3-3bf869\"/\"premium-w3-3bf869-kserve-self-signed-certs\"" Nov 04 02:10:41.779515 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.779497 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:10:41.824875 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.824842 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.824893 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.824925 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.824960 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.824994 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jt47f\" (UniqueName: \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.825040 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825381 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.825359 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825439 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.825390 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.825644 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.825622 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.827654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.827629 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.827838 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.827819 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.886222 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.886192 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jt47f\" (UniqueName: \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\") pod \"premium-w3-3bf869-kserve-5f6b8fc7-wzqrq\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:41.985779 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:41.985750 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:42.376392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:42.376191 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:10:42.379139 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:10:42.379107 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podfa4e95d2_a4e3_4813_8e01_25a0d07cebf6.slice/crio-967019aa98ae0b8f9f6186e5b194007956755e7c66e76ed70488ac44457a3776 WatchSource:0}: Error finding container 967019aa98ae0b8f9f6186e5b194007956755e7c66e76ed70488ac44457a3776: Status 404 returned error can't find the container with id 967019aa98ae0b8f9f6186e5b194007956755e7c66e76ed70488ac44457a3776 Nov 04 02:10:43.079460 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:43.079426 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" event={"ID":"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6","Type":"ContainerStarted","Data":"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570"} Nov 04 02:10:43.079836 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:43.079498 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" event={"ID":"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6","Type":"ContainerStarted","Data":"967019aa98ae0b8f9f6186e5b194007956755e7c66e76ed70488ac44457a3776"} Nov 04 02:10:43.197687 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:43.197640 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" podStartSLOduration=2.197627009 podStartE2EDuration="2.197627009s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:10:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:10:43.195827305 +0000 UTC m=+875.689152469" watchObservedRunningTime="2026-11-04 02:10:43.197627009 +0000 UTC m=+875.690952150" Nov 04 02:10:43.986577 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:43.986544 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:51.986282 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:51.986249 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:51.998948 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:51.998927 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:52.130560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:52.130533 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:10:53.184417 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:10:53.184375 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" podUID="3c019a8a489976b5a638075ae287a9c6" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": context deadline exceeded" Nov 04 02:11:04.503274 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:11:04.503226 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:11:25.893570 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.893494 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:11:25.896793 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.896777 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:25.998845 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.998812 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"kube-root-ca.crt\"" Nov 04 02:11:25.998999 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.998872 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"premium-w1-90ad0f-kserve-self-signed-certs\"" Nov 04 02:11:25.998999 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.998904 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"openshift-service-ca.crt\"" Nov 04 02:11:25.998999 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:25.998913 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w1-90ad0f\"/\"default-dockercfg-zrd8x\"" Nov 04 02:11:26.000285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.000266 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:11:26.036570 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036541 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.036570 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036572 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.036747 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036591 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.036747 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036612 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.036839 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036813 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqqbm\" (UniqueName: \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.036979 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.036844 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.137651 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137616 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.137824 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137663 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.137824 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137691 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.137824 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137722 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.137824 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137798 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqqbm\" (UniqueName: \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.138028 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.137825 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.138087 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.138045 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.138087 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.138062 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.138185 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.138130 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.140188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.140166 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.140391 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.140376 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.199108 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.199039 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqqbm\" (UniqueName: \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\") pod \"premium-w1-90ad0f-kserve-86d65dff79-v8v6k\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.207828 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.207809 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:26.590212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:26.590190 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:11:26.592859 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:11:26.592828 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod00d79ba6_79aa_4603_afdb_56edf41d8b8e.slice/crio-e1ac676ea212597b475b5ad52229ca9766ee291c16bf3b05241244b9251cafca WatchSource:0}: Error finding container e1ac676ea212597b475b5ad52229ca9766ee291c16bf3b05241244b9251cafca: Status 404 returned error can't find the container with id e1ac676ea212597b475b5ad52229ca9766ee291c16bf3b05241244b9251cafca Nov 04 02:11:27.287010 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:27.286978 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" event={"ID":"00d79ba6-79aa-4603-afdb-56edf41d8b8e","Type":"ContainerStarted","Data":"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308"} Nov 04 02:11:27.287381 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:27.287021 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" event={"ID":"00d79ba6-79aa-4603-afdb-56edf41d8b8e","Type":"ContainerStarted","Data":"e1ac676ea212597b475b5ad52229ca9766ee291c16bf3b05241244b9251cafca"} Nov 04 02:11:27.404624 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:27.404566 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" podStartSLOduration=2.4045491549999998 podStartE2EDuration="2.404549155s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:27.403619624 +0000 UTC m=+919.896944779" watchObservedRunningTime="2026-11-04 02:11:27.404549155 +0000 UTC m=+919.897874296" Nov 04 02:11:28.208074 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:28.208046 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:33.673253 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.673215 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:11:33.678439 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.678375 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.783292 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.783265 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"premium-w2-29e7e8-kserve-self-signed-certs\"" Nov 04 02:11:33.784662 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.784642 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:11:33.804336 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804316 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.804393 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804340 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.804393 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804376 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.804499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804464 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.804537 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804514 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.804537 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.804531 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vff9c\" (UniqueName: \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905128 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905095 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905128 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905131 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vff9c\" (UniqueName: \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905212 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905238 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905289 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905353 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905652 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905590 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905720 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905705 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.905809 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.905785 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.907715 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.907692 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.908055 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.908036 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.967010 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.966949 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vff9c\" (UniqueName: \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\") pod \"premium-w2-29e7e8-kserve-5c989d5796-5d45q\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:33.992183 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:33.992158 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:34.371303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:34.371282 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:11:34.373752 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:11:34.373725 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod623f84f2_fa46_494a_9ae0_6d20d5660346.slice/crio-a93147640ea27da5e290e51b4b2f627bd804a6709ffff40ec42ab45942b3eb65 WatchSource:0}: Error finding container a93147640ea27da5e290e51b4b2f627bd804a6709ffff40ec42ab45942b3eb65: Status 404 returned error can't find the container with id a93147640ea27da5e290e51b4b2f627bd804a6709ffff40ec42ab45942b3eb65 Nov 04 02:11:34.502430 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:11:34.502396 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:11:35.328681 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:35.328646 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" event={"ID":"623f84f2-fa46-494a-9ae0-6d20d5660346","Type":"ContainerStarted","Data":"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63"} Nov 04 02:11:35.329101 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:35.328693 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" event={"ID":"623f84f2-fa46-494a-9ae0-6d20d5660346","Type":"ContainerStarted","Data":"a93147640ea27da5e290e51b4b2f627bd804a6709ffff40ec42ab45942b3eb65"} Nov 04 02:11:35.445852 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:35.445801 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" podStartSLOduration=2.4457869580000002 podStartE2EDuration="2.445786958s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:11:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:11:35.444547023 +0000 UTC m=+927.937872166" watchObservedRunningTime="2026-11-04 02:11:35.445786958 +0000 UTC m=+927.939112098" Nov 04 02:11:35.992591 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:35.992562 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:36.208196 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:36.208159 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:36.220730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:36.220704 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:36.350691 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:36.350602 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:11:43.992290 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:43.992258 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:44.006225 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:44.006197 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:11:44.381150 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:11:44.381065 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:12:04.502309 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:12:04.502272 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:12:34.502181 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:12:34.502137 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:13:04.503534 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:13:04.503437 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:13:34.504844 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:13:34.502790 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:13:57.393246 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.393214 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:13:57.396645 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.396627 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436643 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436616 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436775 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436664 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436796 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436892 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436838 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fqh6n\" (UniqueName: \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436927 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436910 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.436973 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.436960 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.498461 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.498431 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"distinct-w2-29e7e8-kserve-self-signed-certs\"" Nov 04 02:13:57.499752 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.499731 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:13:57.537513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537464 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537682 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537588 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537682 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537622 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fqh6n\" (UniqueName: \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537682 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537653 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537856 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537723 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537856 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537757 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.537969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537897 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.538024 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.537965 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.538151 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.538131 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.540227 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.540198 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.540414 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.540395 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.598541 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.598510 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fqh6n\" (UniqueName: \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\") pod \"distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:57.707449 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:57.707367 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:13:58.090368 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:58.090345 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:13:58.092422 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:13:58.092397 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod243ca222_f30f_4837_8651_b489a840dd5b.slice/crio-293c2948d4b403c9fa9b03c2b1e7050e6983fcfd65de7d511973d6ed2488011e WatchSource:0}: Error finding container 293c2948d4b403c9fa9b03c2b1e7050e6983fcfd65de7d511973d6ed2488011e: Status 404 returned error can't find the container with id 293c2948d4b403c9fa9b03c2b1e7050e6983fcfd65de7d511973d6ed2488011e Nov 04 02:13:58.970286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:58.970255 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" event={"ID":"243ca222-f30f-4837-8651-b489a840dd5b","Type":"ContainerStarted","Data":"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc"} Nov 04 02:13:58.970773 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:58.970297 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" event={"ID":"243ca222-f30f-4837-8651-b489a840dd5b","Type":"ContainerStarted","Data":"293c2948d4b403c9fa9b03c2b1e7050e6983fcfd65de7d511973d6ed2488011e"} Nov 04 02:13:59.087338 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:59.087293 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" podStartSLOduration=2.087280219 podStartE2EDuration="2.087280219s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:13:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:13:59.086120938 +0000 UTC m=+1071.579446090" watchObservedRunningTime="2026-11-04 02:13:59.087280219 +0000 UTC m=+1071.580605359" Nov 04 02:13:59.708428 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:13:59.708396 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:14:04.502073 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:14:04.502028 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:14:07.709890 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:07.709859 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:14:07.723584 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:07.723549 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:14:08.027360 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:08.027332 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:14:10.821340 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.821304 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:14:10.826021 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.826005 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.878292 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.878262 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w1-90ad0f-openshift-default-dockercfg-nz7vm\"" Nov 04 02:14:10.892699 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892665 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.892826 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892707 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.892826 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892731 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.892826 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892812 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.892925 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892841 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.892979 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892961 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.893015 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.892989 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hn7ch\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.893047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.893023 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.893093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.893056 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.929939 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.929907 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:14:10.994210 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994179 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994221 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994299 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994325 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hn7ch\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994373 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994735 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994419 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994735 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994513 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994735 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994546 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994735 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994572 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994935 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994779 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994935 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994868 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.994935 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.994925 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.995180 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.995156 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.995286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.995252 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.996624 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.996604 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:10.996931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:10.996913 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:11.055715 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.055682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:11.055866 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.055816 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hn7ch\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:11.138541 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.138450 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:11.566303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.566274 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:14:11.568855 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:14:11.568822 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3b25176b_0f1c_4cae_ad1c_ee9699d253fd.slice/crio-9528bb005fa0dc1cb9e2973f2d531590974bc557b4e6bf878d9e12ee1a14dabe WatchSource:0}: Error finding container 9528bb005fa0dc1cb9e2973f2d531590974bc557b4e6bf878d9e12ee1a14dabe: Status 404 returned error can't find the container with id 9528bb005fa0dc1cb9e2973f2d531590974bc557b4e6bf878d9e12ee1a14dabe Nov 04 02:14:11.571253 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.571218 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:11.571346 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.571286 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:11.571346 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:11.571314 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:12.036427 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:12.036393 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" event={"ID":"3b25176b-0f1c-4cae-ad1c-ee9699d253fd","Type":"ContainerStarted","Data":"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54"} Nov 04 02:14:12.036834 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:12.036443 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" event={"ID":"3b25176b-0f1c-4cae-ad1c-ee9699d253fd","Type":"ContainerStarted","Data":"9528bb005fa0dc1cb9e2973f2d531590974bc557b4e6bf878d9e12ee1a14dabe"} Nov 04 02:14:12.144005 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:12.143972 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:12.160626 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:12.160590 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:12.160930 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:12.160896 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:13.158964 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:13.158924 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:13.159593 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:13.159560 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:14.142500 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:14.140848 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:14.142500 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:14.140911 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:15.145528 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:15.142930 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:15.145528 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:15.143009 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:16.143003 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.142904 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:16.143269 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.143242 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:16.423138 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.423025 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podStartSLOduration=6.423011468 podStartE2EDuration="6.423011468s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:10 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:12.15949859 +0000 UTC m=+1084.652823741" watchObservedRunningTime="2026-11-04 02:14:16.423011468 +0000 UTC m=+1088.916336608" Nov 04 02:14:16.423661 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.423316 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:14:16.426969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.426945 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.482785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.482750 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"kube-root-ca.crt\"" Nov 04 02:14:16.529803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.529765 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"openshift-service-ca.crt\"" Nov 04 02:14:16.529803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.529791 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"default-dockercfg-758fc\"" Nov 04 02:14:16.530081 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.529845 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w4-196e7e\"/\"distinct-w4-196e7e-kserve-self-signed-certs\"" Nov 04 02:14:16.531588 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.531562 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:14:16.567553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567513 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.567757 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567561 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.567757 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567604 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.567757 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567633 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.567928 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567810 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.567989 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.567964 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wqj2b\" (UniqueName: \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670054 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670107 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670150 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670183 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670215 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670302 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wqj2b\" (UniqueName: \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670591 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670549 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.670944 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.670919 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.671529 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.671399 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.673125 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.673105 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.673458 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.673413 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.732316 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.731753 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wqj2b\" (UniqueName: \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\") pod \"distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:16.738960 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:16.738934 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:17.151240 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:17.150811 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" start-of-body= Nov 04 02:14:17.151240 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:17.150868 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.37:15021/healthz/ready\": dial tcp 10.133.0.37:15021: connect: connection refused" Nov 04 02:14:17.158335 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:17.158303 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:14:17.166819 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:14:17.166791 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5c756c97_11b8_48f7_9a31_32e7a5f51ae3.slice/crio-d3e7fb9572f81208e75a5e82a51fc0d1873aeba6c6ab961044ffd50ce64e1f77 WatchSource:0}: Error finding container d3e7fb9572f81208e75a5e82a51fc0d1873aeba6c6ab961044ffd50ce64e1f77: Status 404 returned error can't find the container with id d3e7fb9572f81208e75a5e82a51fc0d1873aeba6c6ab961044ffd50ce64e1f77 Nov 04 02:14:18.089454 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.089418 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" event={"ID":"5c756c97-11b8-48f7-9a31-32e7a5f51ae3","Type":"ContainerStarted","Data":"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73"} Nov 04 02:14:18.089861 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.089486 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" event={"ID":"5c756c97-11b8-48f7-9a31-32e7a5f51ae3","Type":"ContainerStarted","Data":"d3e7fb9572f81208e75a5e82a51fc0d1873aeba6c6ab961044ffd50ce64e1f77"} Nov 04 02:14:18.207297 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.207237 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" podStartSLOduration=2.20722046 podStartE2EDuration="2.20722046s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:18.205585767 +0000 UTC m=+1090.698910922" watchObservedRunningTime="2026-11-04 02:14:18.20722046 +0000 UTC m=+1090.700545600" Nov 04 02:14:18.417272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.417184 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:18.418541 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.418514 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:18.419124 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.419104 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:14:18.739191 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:18.739163 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:22.331218 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:22.331185 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:14:22.331896 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:22.331422 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" containerID="cri-o://a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4" gracePeriod=30 Nov 04 02:14:26.738917 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:26.738887 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:26.751380 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:26.751357 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:27.145771 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.145690 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:14:27.582482 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.582443 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:14:27.781408 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781381 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781427 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781449 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781495 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781548 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781579 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781619 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781651 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781697 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\") pod \"310608eb-1844-4900-ace6-4ccfd0ee7d86\" (UID: \"310608eb-1844-4900-ace6-4ccfd0ee7d86\") " Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781725 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:14:27.781858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781783 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:14:27.782426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781859 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:14:27.782426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.781908 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:14:27.782426 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.782196 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:14:27.783401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.783111 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.783401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.783135 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.783401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.783149 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.783401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.783162 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.783401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.783176 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/310608eb-1844-4900-ace6-4ccfd0ee7d86-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.784632 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.784612 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:14:27.784714 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.784638 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:14:27.784759 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.784722 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "kube-api-access-bc9j2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:14:27.785176 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.785160 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy" pod "310608eb-1844-4900-ace6-4ccfd0ee7d86" (UID: "310608eb-1844-4900-ace6-4ccfd0ee7d86"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:14:27.884660 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.884633 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.884660 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.884656 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.884660 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.884665 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:27.884866 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:27.884676 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bc9j2\" (UniqueName: \"kubernetes.io/projected/310608eb-1844-4900-ace6-4ccfd0ee7d86-kube-api-access-bc9j2\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:14:28.140272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.140184 5029 generic.go:356] "Generic (PLEG): container finished" podID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerID="a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4" exitCode=0 Nov 04 02:14:28.140272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.140234 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" Nov 04 02:14:28.140272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.140266 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" event={"ID":"310608eb-1844-4900-ace6-4ccfd0ee7d86","Type":"ContainerDied","Data":"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4"} Nov 04 02:14:28.140583 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.140315 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk" event={"ID":"310608eb-1844-4900-ace6-4ccfd0ee7d86","Type":"ContainerDied","Data":"0e101b963451ec276ccef3b3dfbfb29cec81f8bc52138b265bc06cfcde52584b"} Nov 04 02:14:28.140583 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.140338 5029 scope.go:122] "RemoveContainer" containerID="a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4" Nov 04 02:14:28.149730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.149711 5029 scope.go:122] "RemoveContainer" containerID="a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4" Nov 04 02:14:28.149982 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:14:28.149961 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4\": container with ID starting with a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4 not found: ID does not exist" containerID="a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4" Nov 04 02:14:28.150042 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.149994 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4"} err="failed to get container status \"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4\": rpc error: code = NotFound desc = could not find container \"a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4\": container with ID starting with a496716f55cbcdd3cbfe3100f4e64c9428e8a06a6473b7465347076e246beaf4 not found: ID does not exist" Nov 04 02:14:28.303730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.303692 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:14:28.353643 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:28.353614 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-jchrk"] Nov 04 02:14:29.620050 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:29.620018 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" path="/var/lib/kubelet/pods/310608eb-1844-4900-ace6-4ccfd0ee7d86/volumes" Nov 04 02:14:34.503284 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:14:34.503240 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:14:50.838286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.838250 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:14:50.838893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.838643 5029 cpu_manager.go:414] "containerMap: removing container" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" Nov 04 02:14:50.838893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.838654 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" Nov 04 02:14:50.838893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.838728 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="310608eb-1844-4900-ace6-4ccfd0ee7d86" containerName="istio-proxy" Nov 04 02:14:50.842783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.842766 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:50.895186 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.895156 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-24bea9-openshift-default-dockercfg-g5z8q\"" Nov 04 02:14:50.946627 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:50.946595 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:14:51.004400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004371 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004577 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004416 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldgtq\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004577 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004541 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004577 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004561 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004743 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004591 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004743 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004703 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004824 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004757 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004856 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004836 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.004907 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.004893 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105530 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105442 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105530 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105494 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105530 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105528 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105555 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105589 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105620 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105686 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.105754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105736 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106029 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105786 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldgtq\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106029 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.105869 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106029 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.106015 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106155 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.106041 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106186 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.106163 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.106269 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.106246 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.107858 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.107835 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.108411 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.108384 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.169250 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.169219 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.169540 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.169520 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldgtq\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.455366 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.455270 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:51.879910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.879883 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:14:51.881775 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:14:51.881740 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod434d0892_a3a8_44a8_a187_765bbd920700.slice/crio-2549cc8c97a81dc9bc1e25964c2dc1bb7cfc019263aa4beb3344fc49bea20f02 WatchSource:0}: Error finding container 2549cc8c97a81dc9bc1e25964c2dc1bb7cfc019263aa4beb3344fc49bea20f02: Status 404 returned error can't find the container with id 2549cc8c97a81dc9bc1e25964c2dc1bb7cfc019263aa4beb3344fc49bea20f02 Nov 04 02:14:51.883996 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.883965 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:51.884058 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.884028 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:51.884058 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:51.884055 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:14:52.257707 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:52.257662 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" event={"ID":"434d0892-a3a8-44a8-a187-765bbd920700","Type":"ContainerStarted","Data":"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92"} Nov 04 02:14:52.257914 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:52.257728 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" event={"ID":"434d0892-a3a8-44a8-a187-765bbd920700","Type":"ContainerStarted","Data":"2549cc8c97a81dc9bc1e25964c2dc1bb7cfc019263aa4beb3344fc49bea20f02"} Nov 04 02:14:52.462100 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:52.457806 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:52.465622 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:52.465588 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:52.465788 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:52.465649 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:53.459492 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:53.458790 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:53.459492 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:53.458847 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:54.458313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:54.457865 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:54.458313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:54.457926 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:55.458613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:55.458576 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:55.459571 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:55.459153 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:56.459491 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:56.458817 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:56.459491 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:56.458876 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:57.455857 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:57.455822 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:57.456122 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:57.456099 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:58.456176 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:58.456142 5029 patch_prober.go:29] interesting pod/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" start-of-body= Nov 04 02:14:58.456573 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:58.456194 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.39:15021/healthz/ready\": dial tcp 10.133.0.39:15021: connect: connection refused" Nov 04 02:14:59.460700 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:59.460673 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:59.461062 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:59.460929 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:59.461751 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:59.461729 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:14:59.631553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:14:59.631509 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podStartSLOduration=9.631496344 podStartE2EDuration="9.631496344s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:14:50 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:14:52.384189638 +0000 UTC m=+1124.877514782" watchObservedRunningTime="2026-11-04 02:14:59.631496344 +0000 UTC m=+1132.124821483" Nov 04 02:15:00.176652 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.176623 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:00.179844 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.179822 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.212225 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.212197 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmklv\" (UniqueName: \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.212369 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.212354 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.232549 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.232525 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9kb5w\"" Nov 04 02:15:00.274249 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.274219 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:00.313432 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.313405 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.313629 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.313440 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmklv\" (UniqueName: \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.314096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.314077 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.377974 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.377941 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmklv\" (UniqueName: \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\") pod \"maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.493253 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.493221 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:00.864849 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.864806 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:00.867558 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:15:00.867532 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1f4ee692_b950_4e1a_9cd9_6a94ccdcd756.slice/crio-de5f06e43d9c31ae8dcb733008541818c83b351217310c5365ad8940b1d5610e WatchSource:0}: Error finding container de5f06e43d9c31ae8dcb733008541818c83b351217310c5365ad8940b1d5610e: Status 404 returned error can't find the container with id de5f06e43d9c31ae8dcb733008541818c83b351217310c5365ad8940b1d5610e Nov 04 02:15:00.869301 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:00.869280 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:15:01.311642 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:01.311607 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" event={"ID":"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756","Type":"ContainerStarted","Data":"de5f06e43d9c31ae8dcb733008541818c83b351217310c5365ad8940b1d5610e"} Nov 04 02:15:04.506216 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:15:04.506108 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:15:06.337798 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:06.337699 5029 generic.go:356] "Generic (PLEG): container finished" podID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerID="021767ad1c8bc1c323a49a76308d1e658e72f792de1c35177243aae09789d8ba" exitCode=60 Nov 04 02:15:06.338155 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:06.337786 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" event={"ID":"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756","Type":"ContainerDied","Data":"021767ad1c8bc1c323a49a76308d1e658e72f792de1c35177243aae09789d8ba"} Nov 04 02:15:06.338155 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:06.338028 5029 scope.go:122] "RemoveContainer" containerID="021767ad1c8bc1c323a49a76308d1e658e72f792de1c35177243aae09789d8ba" Nov 04 02:15:07.345531 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:07.345495 5029 generic.go:356] "Generic (PLEG): container finished" podID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerID="c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020" exitCode=60 Nov 04 02:15:07.345915 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:07.345571 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" event={"ID":"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756","Type":"ContainerDied","Data":"c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020"} Nov 04 02:15:07.345915 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:07.345623 5029 scope.go:122] "RemoveContainer" containerID="021767ad1c8bc1c323a49a76308d1e658e72f792de1c35177243aae09789d8ba" Nov 04 02:15:07.345915 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:07.345875 5029 scope.go:122] "RemoveContainer" containerID="c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020" Nov 04 02:15:07.346133 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:15:07.346113 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc_odh-ai-gateway-infra(1f4ee692-b950-4e1a-9cd9-6a94ccdcd756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" Nov 04 02:15:08.352140 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:08.352116 5029 scope.go:122] "RemoveContainer" containerID="c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020" Nov 04 02:15:08.352539 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:15:08.352352 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 10s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc_odh-ai-gateway-infra(1f4ee692-b950-4e1a-9cd9-6a94ccdcd756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" Nov 04 02:15:17.615920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:17.615882 5029 scope.go:122] "RemoveContainer" containerID="c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020" Nov 04 02:15:18.398183 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:18.398158 5029 generic.go:356] "Generic (PLEG): container finished" podID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerID="56f1ce2dcfa21dd42ddc5e7600913806cc9dd95024ceda28a7527c74e6bc28aa" exitCode=60 Nov 04 02:15:18.398306 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:18.398232 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" event={"ID":"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756","Type":"ContainerDied","Data":"56f1ce2dcfa21dd42ddc5e7600913806cc9dd95024ceda28a7527c74e6bc28aa"} Nov 04 02:15:18.398306 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:18.398286 5029 scope.go:122] "RemoveContainer" containerID="c046863198803682b4fd2cfc59bfb90b708b80e05be312a20ffde5422c7f7020" Nov 04 02:15:18.398658 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:18.398640 5029 scope.go:122] "RemoveContainer" containerID="56f1ce2dcfa21dd42ddc5e7600913806cc9dd95024ceda28a7527c74e6bc28aa" Nov 04 02:15:18.398869 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:15:18.398852 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"cleanup\" with CrashLoopBackOff: \"back-off 20s restarting failed container=cleanup pod=maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc_odh-ai-gateway-infra(1f4ee692-b950-4e1a-9cd9-6a94ccdcd756)\"" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" Nov 04 02:15:19.522776 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.522745 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:19.662522 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.662500 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:19.797313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.797237 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\") pod \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " Nov 04 02:15:19.797313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.797300 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\" (UniqueName: \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\") pod \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\" (UID: \"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756\") " Nov 04 02:15:19.797581 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.797557 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca" pod "1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" (UID: "1f4ee692-b950-4e1a-9cd9-6a94ccdcd756"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:15:19.797658 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.797643 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-maas-api-service-ca\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:15:19.799647 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.799621 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv" pod "1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" (UID: "1f4ee692-b950-4e1a-9cd9-6a94ccdcd756"). InnerVolumeSpecName "kube-api-access-rmklv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:15:19.898741 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:19.898714 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmklv\" (UniqueName: \"kubernetes.io/projected/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756-kube-api-access-rmklv\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:15:20.410465 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:20.410432 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" event={"ID":"1f4ee692-b950-4e1a-9cd9-6a94ccdcd756","Type":"ContainerDied","Data":"de5f06e43d9c31ae8dcb733008541818c83b351217310c5365ad8940b1d5610e"} Nov 04 02:15:20.410465 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:20.410456 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc" Nov 04 02:15:20.410688 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:20.410539 5029 scope.go:122] "RemoveContainer" containerID="56f1ce2dcfa21dd42ddc5e7600913806cc9dd95024ceda28a7527c74e6bc28aa" Nov 04 02:15:20.574523 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:20.574493 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:20.624200 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:20.624165 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-key-cleanup-e2e-worker-w1-90ad0f-29895975-zx7vc"] Nov 04 02:15:21.619229 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:21.619201 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" path="/var/lib/kubelet/pods/1f4ee692-b950-4e1a-9cd9-6a94ccdcd756/volumes" Nov 04 02:15:33.686988 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.686954 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687280 5029 cpu_manager.go:414] "containerMap: removing container" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687290 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687322 5029 cpu_manager.go:414] "containerMap: removing container" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687328 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687386 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.687412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.687398 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:33.690706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.690688 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.792432 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.792401 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"e2e-models-e2e-worker-w2-29e7e8\"/\"distinct-2-w2-29e7e8-kserve-self-signed-certs\"" Nov 04 02:15:33.793958 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.793938 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:15:33.829287 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829256 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.829287 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829286 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.829511 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829322 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.829511 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829402 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.829583 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829530 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcq8t\" (UniqueName: \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.829615 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.829592 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930161 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930126 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930363 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930227 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930363 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930249 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930363 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930294 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930363 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930323 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930363 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930348 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcq8t\" (UniqueName: \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930640 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930571 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930699 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930638 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.930789 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.930764 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.932535 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.932507 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.932889 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.932871 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:33.994178 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:33.994150 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcq8t\" (UniqueName: \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\") pod \"distinct-2-w2-29e7e8-kserve-945878948-92pg9\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:34.001843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:34.001819 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:34.381186 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:34.381163 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:15:34.383896 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:15:34.383871 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78ebac77_af85_4c16_a267_0dbc95331e03.slice/crio-5a0b322770307ab0a8d07d456452ff35ca9670a8d0748bcbcb080e6ce4a9a218 WatchSource:0}: Error finding container 5a0b322770307ab0a8d07d456452ff35ca9670a8d0748bcbcb080e6ce4a9a218: Status 404 returned error can't find the container with id 5a0b322770307ab0a8d07d456452ff35ca9670a8d0748bcbcb080e6ce4a9a218 Nov 04 02:15:34.476170 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:34.476128 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" event={"ID":"78ebac77-af85-4c16-a267-0dbc95331e03","Type":"ContainerStarted","Data":"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5"} Nov 04 02:15:34.476357 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:34.476184 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" event={"ID":"78ebac77-af85-4c16-a267-0dbc95331e03","Type":"ContainerStarted","Data":"5a0b322770307ab0a8d07d456452ff35ca9670a8d0748bcbcb080e6ce4a9a218"} Nov 04 02:15:34.503157 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:15:34.503122 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:15:34.593646 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:34.593596 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" podStartSLOduration=1.593581245 podStartE2EDuration="1.593581245s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:33 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:34.592001727 +0000 UTC m=+1167.085326866" watchObservedRunningTime="2026-11-04 02:15:34.593581245 +0000 UTC m=+1167.086906384" Nov 04 02:15:35.002424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:35.002395 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:41.547927 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.547894 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:15:41.548302 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.548217 5029 cpu_manager.go:414] "containerMap: removing container" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:41.548302 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.548227 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:41.548371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.548313 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="1f4ee692-b950-4e1a-9cd9-6a94ccdcd756" containerName="cleanup" Nov 04 02:15:41.555687 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.555667 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612065 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612104 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612123 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612280 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612244 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612280 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612272 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612340 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612292 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4r2hk\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612374 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612347 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612419 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612406 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.612543 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.612531 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.646878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.646853 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:15:41.713243 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713212 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713379 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713249 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713379 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713276 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4r2hk\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713379 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713308 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713578 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713392 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713637 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713605 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713692 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713640 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713692 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713675 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713786 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713701 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713786 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713710 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713786 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713731 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.713897 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.713813 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.714047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.714016 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.714308 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.714283 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.716001 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.715982 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.716114 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.716096 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.773979 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.773962 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.774160 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.774144 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4r2hk\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\") pod \"e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:41.869143 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:41.869090 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:42.294096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.294061 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:15:42.296873 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:15:42.296844 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffb697dc_59d8_4adc_90f9_5528919587ba.slice/crio-2a61a53abd3115434536a375410874835f7489ea5341a6718adbef853c35c6f1 WatchSource:0}: Error finding container 2a61a53abd3115434536a375410874835f7489ea5341a6718adbef853c35c6f1: Status 404 returned error can't find the container with id 2a61a53abd3115434536a375410874835f7489ea5341a6718adbef853c35c6f1 Nov 04 02:15:42.299166 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.299133 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:15:42.299272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.299198 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:15:42.299272 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.299233 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:15:42.514152 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.514118 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" event={"ID":"ffb697dc-59d8-4adc-90f9-5528919587ba","Type":"ContainerStarted","Data":"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c"} Nov 04 02:15:42.514326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.514189 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" event={"ID":"ffb697dc-59d8-4adc-90f9-5528919587ba","Type":"ContainerStarted","Data":"2a61a53abd3115434536a375410874835f7489ea5341a6718adbef853c35c6f1"} Nov 04 02:15:42.869829 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:42.869788 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:43.871239 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:43.871206 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:43.871674 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:43.871267 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:44.002085 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.002059 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:44.014636 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.014610 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:44.134273 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.134178 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podStartSLOduration=3.134165646 podStartE2EDuration="3.134165646s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:15:41 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:15:42.635562334 +0000 UTC m=+1175.128887499" watchObservedRunningTime="2026-11-04 02:15:44.134165646 +0000 UTC m=+1176.627490786" Nov 04 02:15:44.534203 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.534169 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:15:44.869698 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.869608 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:44.869698 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:44.869659 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:45.870348 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:45.870312 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:45.870871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:45.870364 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:46.870548 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:46.870516 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:46.870922 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:46.870561 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:47.871234 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:47.871198 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:47.871750 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:47.871260 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:48.869769 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:48.869736 5029 patch_prober.go:29] interesting pod/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" start-of-body= Nov 04 02:15:48.869947 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:48.869799 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.42:15021/healthz/ready\": dial tcp 10.133.0.42:15021: connect: connection refused" Nov 04 02:15:49.873039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:49.873013 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:49.873504 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:49.873422 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:15:49.874021 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:15:49.874005 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:16:04.501427 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:16:04.501375 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:16:08.362186 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.362155 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:16:08.366910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.366894 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.419177 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.419144 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-extmod-a-0508d1-serving-cert\"" Nov 04 02:16:08.425792 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.425767 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:16:08.496741 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.496710 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.496902 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.496747 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.496902 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.496772 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.496902 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.496891 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qdhks\" (UniqueName: \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.598341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.598301 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qdhks\" (UniqueName: \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.598560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.598429 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.598560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.598452 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.598560 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.598506 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.598740 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:16:08.598614 5029 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-serving-cert: secret "maas-api-e2e-extmod-a-0508d1-serving-cert" not found Nov 04 02:16:08.598740 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:16:08.598662 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls podName:7d74fa53-825d-4734-b4db-4dc70459cd6a nodeName:}" failed. No retries permitted until 2026-11-04 02:16:09.098644009 +0000 UTC m=+1201.591969128 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls") pod "maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" (UID: "7d74fa53-825d-4734-b4db-4dc70459cd6a") : secret "maas-api-e2e-extmod-a-0508d1-serving-cert" not found Nov 04 02:16:08.599202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.599180 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.600967 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.600946 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:08.659772 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:08.659696 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qdhks\" (UniqueName: \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:09.104172 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:09.104140 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:09.106664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:09.106636 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") pod \"maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:09.279809 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:09.279775 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:09.653255 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:09.653226 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:16:09.655300 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:16:09.655275 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d74fa53_825d_4734_b4db_4dc70459cd6a.slice/crio-cf030fb572ba298205cc43ce1544d90aeb0dcec138d0f8994d872f2109439ccf WatchSource:0}: Error finding container cf030fb572ba298205cc43ce1544d90aeb0dcec138d0f8994d872f2109439ccf: Status 404 returned error can't find the container with id cf030fb572ba298205cc43ce1544d90aeb0dcec138d0f8994d872f2109439ccf Nov 04 02:16:10.648944 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:10.648904 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" event={"ID":"7d74fa53-825d-4734-b4db-4dc70459cd6a","Type":"ContainerStarted","Data":"cf030fb572ba298205cc43ce1544d90aeb0dcec138d0f8994d872f2109439ccf"} Nov 04 02:16:11.655070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:11.655031 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" event={"ID":"7d74fa53-825d-4734-b4db-4dc70459cd6a","Type":"ContainerStarted","Data":"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0"} Nov 04 02:16:11.655539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:11.655136 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:11.819031 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:11.818974 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" podStartSLOduration=2.312554637 podStartE2EDuration="3.818959147s" totalImagesPullingTime="1.50640451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:11.818064369 +0000 UTC m=+1204.311389510" watchObservedRunningTime="2026-11-04 02:16:11.818959147 +0000 UTC m=+1204.312284284" Nov 04 02:16:17.665421 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:17.665392 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:16:18.761039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.761010 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:16:18.764725 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.764710 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.817347 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.817326 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\"" Nov 04 02:16:18.868780 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.868756 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:16:18.908373 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908340 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908382 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908408 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908638 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908713 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908757 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908798 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wm8dv\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.908843 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908834 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:18.909222 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:18.908904 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010524 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010493 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010555 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010593 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wm8dv\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010632 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010706 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010700 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010784 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010812 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010836 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010879 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.010926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.010908 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.011212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.011122 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.011279 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.011243 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.011336 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.011294 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.011388 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.011339 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.013248 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.013217 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.013448 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.013428 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.071872 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.071851 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.072043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.072021 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wm8dv\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.076820 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.076801 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:19.361732 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.361659 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:16:19.366145 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.366128 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.413948 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.413903 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414075 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414022 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414075 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414047 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414179 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414135 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414179 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414150 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fl2vx\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414304 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414283 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414349 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414325 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414349 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414346 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.414419 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.414363 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.418406 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.418384 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-196e7e-openshift-default-dockercfg-cdj7c\"" Nov 04 02:16:19.427007 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.426983 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:16:19.514988 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.514958 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515148 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515000 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515148 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515032 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515148 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515059 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515148 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515091 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515157 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515181 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515279 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515302 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fl2vx\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.515621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.515594 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.516100 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.516076 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.516270 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.516178 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.516385 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.516266 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.516566 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.516540 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.517660 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.517637 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.517865 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.517850 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.527549 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.527526 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:16:19.531197 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:16:19.531175 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod418ac499_1567_4c0d_a745_778881063cb2.slice/crio-df18413e5b764163c50fdabca77a52cb54485544bdeeefa8f6aac02dd635d874 WatchSource:0}: Error finding container df18413e5b764163c50fdabca77a52cb54485544bdeeefa8f6aac02dd635d874: Status 404 returned error can't find the container with id df18413e5b764163c50fdabca77a52cb54485544bdeeefa8f6aac02dd635d874 Nov 04 02:16:19.533570 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.533541 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:19.533654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.533601 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:19.533654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.533629 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:19.575950 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.575925 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.576052 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.576037 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fl2vx\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.679535 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.679324 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:19.702096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.699096 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" event={"ID":"418ac499-1567-4c0d-a745-778881063cb2","Type":"ContainerStarted","Data":"41e5015bbc9c9de34d2929796aa0900ee3a6e46171c620e5d33ce38bb075b6b8"} Nov 04 02:16:19.702096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:19.699178 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" event={"ID":"418ac499-1567-4c0d-a745-778881063cb2","Type":"ContainerStarted","Data":"df18413e5b764163c50fdabca77a52cb54485544bdeeefa8f6aac02dd635d874"} Nov 04 02:16:20.077929 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.077899 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:20.078995 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.078970 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:20.079097 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.079019 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:20.110691 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.110622 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podStartSLOduration=2.110607972 podStartE2EDuration="2.110607972s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:19.845543794 +0000 UTC m=+1212.338868956" watchObservedRunningTime="2026-11-04 02:16:20.110607972 +0000 UTC m=+1212.603933111" Nov 04 02:16:20.111412 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.111395 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:16:20.112997 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:16:20.112973 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf84592a2_efda_4807_b2a5_57a6a9d0b48f.slice/crio-f2bb8e9c81bd670b5dbfd63e7c45c6857c517196d3113b6da2a0e84c7215334c WatchSource:0}: Error finding container f2bb8e9c81bd670b5dbfd63e7c45c6857c517196d3113b6da2a0e84c7215334c: Status 404 returned error can't find the container with id f2bb8e9c81bd670b5dbfd63e7c45c6857c517196d3113b6da2a0e84c7215334c Nov 04 02:16:20.115450 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.115420 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:20.115582 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.115513 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:20.115582 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.115557 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:16:20.711561 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.710250 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" event={"ID":"f84592a2-efda-4807-b2a5-57a6a9d0b48f","Type":"ContainerStarted","Data":"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d"} Nov 04 02:16:20.711561 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.710322 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" event={"ID":"f84592a2-efda-4807-b2a5-57a6a9d0b48f","Type":"ContainerStarted","Data":"f2bb8e9c81bd670b5dbfd63e7c45c6857c517196d3113b6da2a0e84c7215334c"} Nov 04 02:16:20.832179 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:20.832128 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podStartSLOduration=1.832113356 podStartE2EDuration="1.832113356s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:16:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:16:20.830353198 +0000 UTC m=+1213.323678363" watchObservedRunningTime="2026-11-04 02:16:20.832113356 +0000 UTC m=+1213.325438495" Nov 04 02:16:21.078120 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:21.078085 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:21.078553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:21.078137 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:21.685496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:21.684510 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:21.686061 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:21.686031 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:21.686293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:21.686265 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:22.078495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:22.077881 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:22.078495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:22.077959 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:22.680933 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:22.680898 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:22.681236 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:22.681186 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:23.078836 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:23.078801 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:23.079454 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:23.079423 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:23.684860 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:23.684823 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:23.685165 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:23.685129 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:24.078872 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:24.078834 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:24.079874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:24.079445 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:24.681871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:24.681831 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:24.682147 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:24.682118 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:25.086281 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:25.085801 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:25.086281 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:25.085862 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:25.680087 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:25.680053 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:25.681164 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:25.681129 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:26.078131 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:26.078086 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" start-of-body= Nov 04 02:16:26.078440 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:26.078399 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.44:15021/healthz/ready\": dial tcp 10.133.0.44:15021: connect: connection refused" Nov 04 02:16:26.680379 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:26.680336 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:26.680878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:26.680403 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:27.585684 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:27.585647 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:27.586075 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:27.586034 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:27.587228 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:27.587208 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:16:27.680558 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:27.680533 5029 patch_prober.go:29] interesting pod/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" start-of-body= Nov 04 02:16:27.680894 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:27.680588 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.45:15021/healthz/ready\": dial tcp 10.133.0.45:15021: connect: connection refused" Nov 04 02:16:28.685023 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:28.684994 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:28.685411 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:28.685392 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:28.686002 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:16:28.685985 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:16:34.503591 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:16:34.503548 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:17:04.503665 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:17:04.503629 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:17:07.547942 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.547901 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:17:07.552126 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.552105 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.609889 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.609854 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-extmod-b-3f76d0-openshift-default-dockercfg-kkjfv\"" Nov 04 02:17:07.662661 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.662630 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:17:07.712857 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.712828 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.712896 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713198 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713175 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbnd5\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713216 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713315 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713278 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713315 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713304 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713325 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713415 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713370 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.713542 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.713440 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.814783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814690 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.814783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814757 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814815 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qbnd5\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814848 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814885 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814910 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814937 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815008 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.814966 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815023 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815136 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815167 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815418 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815464 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.815913 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.815885 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.817510 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.817465 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.817806 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.817785 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.876667 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.876629 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:07.876809 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:07.876709 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbnd5\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\") pod \"e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:08.172385 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.172295 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:08.605788 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.605764 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:17:08.607172 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:17:08.607147 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7730d765_c285_4766_97be_35765e0ea2c2.slice/crio-236b949e743c75eb98840ff40d85482d9504b1d15a5d3c1be6306f08a9168899 WatchSource:0}: Error finding container 236b949e743c75eb98840ff40d85482d9504b1d15a5d3c1be6306f08a9168899: Status 404 returned error can't find the container with id 236b949e743c75eb98840ff40d85482d9504b1d15a5d3c1be6306f08a9168899 Nov 04 02:17:08.609659 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.609631 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:08.609746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.609711 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:08.609746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.609738 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:08.960295 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.960201 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerStarted","Data":"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47"} Nov 04 02:17:08.960452 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:08.960419 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerStarted","Data":"236b949e743c75eb98840ff40d85482d9504b1d15a5d3c1be6306f08a9168899"} Nov 04 02:17:09.081770 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:09.081700 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podStartSLOduration=2.081687124 podStartE2EDuration="2.081687124s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:09.080957486 +0000 UTC m=+1261.574282667" watchObservedRunningTime="2026-11-04 02:17:09.081687124 +0000 UTC m=+1261.575012266" Nov 04 02:17:09.173450 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:09.173411 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:09.179189 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:09.179166 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:09.964982 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:09.964942 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:09.965935 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:09.965912 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:17:27.526656 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:27.526569 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:17:34.505607 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:17:34.505559 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:17:38.746605 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.746564 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj"] Nov 04 02:17:38.750155 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.750133 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.847376 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.847338 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj"] Nov 04 02:17:38.848817 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.848788 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.848988 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.848833 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.848988 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.848951 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849078 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849124 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849182 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849216 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5v6nv\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-kube-api-access-5v6nv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849410 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849386 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.849480 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.849446 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950378 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950344 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950389 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5v6nv\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-kube-api-access-5v6nv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950444 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950500 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950532 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950555 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950581 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950904 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950647 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.950904 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.950674 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.951103 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.951066 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.951238 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.951115 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.951238 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.951133 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.951238 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.951155 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.951582 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.951562 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.953278 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.953254 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:38.953445 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:38.953425 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:39.011670 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.011595 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:39.011803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.011690 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5v6nv\" (UniqueName: \"kubernetes.io/projected/f42723eb-4c4b-490d-82d0-4fa690ee2a8a-kube-api-access-5v6nv\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj\" (UID: \"f42723eb-4c4b-490d-82d0-4fa690ee2a8a\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:39.063673 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.063628 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:39.496165 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.496135 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj"] Nov 04 02:17:39.498144 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:17:39.498119 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf42723eb_4c4b_490d_82d0_4fa690ee2a8a.slice/crio-7128abd04696862e25eca0d38d5684f921f8f2189c4354f73d35a76f89445167 WatchSource:0}: Error finding container 7128abd04696862e25eca0d38d5684f921f8f2189c4354f73d35a76f89445167: Status 404 returned error can't find the container with id 7128abd04696862e25eca0d38d5684f921f8f2189c4354f73d35a76f89445167 Nov 04 02:17:39.500400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.500367 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:39.500494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.500433 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:39.500494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:39.500462 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:40.115150 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:40.115097 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" event={"ID":"f42723eb-4c4b-490d-82d0-4fa690ee2a8a","Type":"ContainerStarted","Data":"f7397f58133dcec46563c1fc9ab71fb243d427bef068ddb1c9119487f100070e"} Nov 04 02:17:40.115585 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:40.115209 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" event={"ID":"f42723eb-4c4b-490d-82d0-4fa690ee2a8a","Type":"ContainerStarted","Data":"7128abd04696862e25eca0d38d5684f921f8f2189c4354f73d35a76f89445167"} Nov 04 02:17:40.238014 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:40.237966 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podStartSLOduration=2.237950471 podStartE2EDuration="2.237950471s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:40.235652455 +0000 UTC m=+1292.728977626" watchObservedRunningTime="2026-11-04 02:17:40.237950471 +0000 UTC m=+1292.731275677" Nov 04 02:17:41.064854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:41.064815 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:41.066627 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:41.066598 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:41.066758 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:41.066657 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:42.065053 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.065018 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:42.065518 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.065096 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:42.563709 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.563654 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" podUID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" containerName="main" containerID="cri-o://4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308" gracePeriod=45 Nov 04 02:17:42.834096 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.834069 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:17:42.902688 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.902657 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\" (UniqueName: \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.902899 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.902757 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.902899 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.902804 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\" (UniqueName: \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.902899 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.902867 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.903047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.902937 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.903047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903021 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\") pod \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\" (UID: \"00d79ba6-79aa-4603-afdb-56edf41d8b8e\") " Nov 04 02:17:42.903154 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903018 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:17:42.903212 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903171 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:17:42.903257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903210 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:17:42.903448 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903425 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:42.903576 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903451 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:42.903576 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.903485 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:42.905279 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.905256 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:17:42.905279 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.905274 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:17:42.905401 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:42.905314 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm" pod "00d79ba6-79aa-4603-afdb-56edf41d8b8e" (UID: "00d79ba6-79aa-4603-afdb-56edf41d8b8e"). InnerVolumeSpecName "kube-api-access-kqqbm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:17:43.004389 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.004357 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/00d79ba6-79aa-4603-afdb-56edf41d8b8e-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:43.004389 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.004387 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/00d79ba6-79aa-4603-afdb-56edf41d8b8e-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:43.004603 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.004399 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kqqbm\" (UniqueName: \"kubernetes.io/projected/00d79ba6-79aa-4603-afdb-56edf41d8b8e-kube-api-access-kqqbm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:17:43.064748 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.064238 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:43.064748 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.064314 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:43.132633 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.132551 5029 generic.go:356] "Generic (PLEG): container finished" podID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" containerID="4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308" exitCode=0 Nov 04 02:17:43.132984 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.132694 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" event={"ID":"00d79ba6-79aa-4603-afdb-56edf41d8b8e","Type":"ContainerDied","Data":"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308"} Nov 04 02:17:43.132984 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.132725 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" event={"ID":"00d79ba6-79aa-4603-afdb-56edf41d8b8e","Type":"ContainerDied","Data":"e1ac676ea212597b475b5ad52229ca9766ee291c16bf3b05241244b9251cafca"} Nov 04 02:17:43.132984 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.132744 5029 scope.go:122] "RemoveContainer" containerID="4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308" Nov 04 02:17:43.132984 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.132863 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k" Nov 04 02:17:43.174780 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.174758 5029 scope.go:122] "RemoveContainer" containerID="4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308" Nov 04 02:17:43.175152 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:17:43.175130 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308\": container with ID starting with 4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308 not found: ID does not exist" containerID="4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308" Nov 04 02:17:43.175256 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.175161 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308"} err="failed to get container status \"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308\": rpc error: code = NotFound desc = could not find container \"4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308\": container with ID starting with 4ae724ee849f7c65235a541eb00efc1a600e5c0ab7874d488cf3529fcdbef308 not found: ID does not exist" Nov 04 02:17:43.319547 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.319513 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:17:43.327184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.327152 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w1-90ad0f/premium-w1-90ad0f-kserve-86d65dff79-v8v6k"] Nov 04 02:17:43.621655 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:43.621620 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" path="/var/lib/kubelet/pods/00d79ba6-79aa-4603-afdb-56edf41d8b8e/volumes" Nov 04 02:17:44.065075 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:44.065018 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:44.065291 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:44.065095 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:45.064742 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:45.064701 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:45.065346 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:45.065312 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:46.064708 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:46.064666 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:46.064939 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:46.064736 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:47.064313 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:47.064279 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" start-of-body= Nov 04 02:17:47.064525 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:47.064337 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": dial tcp 10.133.0.47:15021: connect: connection refused" Nov 04 02:17:48.295986 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:48.295948 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:48.296526 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:48.296286 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:48.297551 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:48.297523 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" Nov 04 02:17:53.892544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:53.892510 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47"] Nov 04 02:17:53.892946 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:53.892934 5029 cpu_manager.go:414] "containerMap: removing container" podUID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" containerName="main" Nov 04 02:17:53.892946 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:53.892944 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" containerName="main" Nov 04 02:17:53.893027 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:53.893017 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="00d79ba6-79aa-4603-afdb-56edf41d8b8e" containerName="main" Nov 04 02:17:53.896072 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:53.896056 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025504 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025298 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025504 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025372 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025504 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025441 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025538 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025595 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-kube-api-access-nv8q7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025655 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025680 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025772 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.025832 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.025803 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d139de1-dfac-49f1-a95d-4f2470c81a89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.098903 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.098868 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47"] Nov 04 02:17:54.126934 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.126890 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127104 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.126954 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127104 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.126987 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-kube-api-access-nv8q7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127104 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127038 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127104 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127071 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127109 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127145 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d139de1-dfac-49f1-a95d-4f2470c81a89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127286 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127514 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127343 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127514 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127459 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-certs\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.127812 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.127784 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-credential-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.128142 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.128117 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-data\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.128314 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.128118 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-workload-socket\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.128613 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.128587 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3d139de1-dfac-49f1-a95d-4f2470c81a89-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.130172 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.130143 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-envoy\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.130690 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.130666 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.188396 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.188324 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-istio-token\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.188594 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.188568 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv8q7\" (UniqueName: \"kubernetes.io/projected/3d139de1-dfac-49f1-a95d-4f2470c81a89-kube-api-access-nv8q7\") pod \"maas-default-gateway-openshift-default-7bdc57c4cf-8rt47\" (UID: \"3d139de1-dfac-49f1-a95d-4f2470c81a89\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.208257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.208225 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:54.635877 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.635849 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47"] Nov 04 02:17:54.637349 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:17:54.637325 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3d139de1_dfac_49f1_a95d_4f2470c81a89.slice/crio-045820c909fcc1f766447778aaded620c7116c2ce4b08493a45038e0e4040485 WatchSource:0}: Error finding container 045820c909fcc1f766447778aaded620c7116c2ce4b08493a45038e0e4040485: Status 404 returned error can't find the container with id 045820c909fcc1f766447778aaded620c7116c2ce4b08493a45038e0e4040485 Nov 04 02:17:54.639773 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.639744 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:54.639830 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.639807 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:54.639866 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:54.639836 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:17:55.227005 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:55.226968 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" event={"ID":"3d139de1-dfac-49f1-a95d-4f2470c81a89","Type":"ContainerStarted","Data":"a19dbf7f63b2537768550d8e1d1f822c064bb71bb0ff48d5e3897de87df8d67e"} Nov 04 02:17:55.227521 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:55.227020 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" event={"ID":"3d139de1-dfac-49f1-a95d-4f2470c81a89","Type":"ContainerStarted","Data":"045820c909fcc1f766447778aaded620c7116c2ce4b08493a45038e0e4040485"} Nov 04 02:17:55.400629 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:55.400576 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podStartSLOduration=2.400562071 podStartE2EDuration="2.400562071s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:17:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:17:55.397952551 +0000 UTC m=+1307.891277692" watchObservedRunningTime="2026-11-04 02:17:55.400562071 +0000 UTC m=+1307.893887190" Nov 04 02:17:56.208973 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:56.208932 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:17:56.210386 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:56.210360 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:17:56.210489 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:56.210415 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:17:57.209083 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:57.209040 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:17:57.209511 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:57.209104 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:17:58.208883 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:58.208846 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:17:58.209044 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:58.208900 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:17:59.209728 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:59.209686 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:17:59.210171 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:17:59.209744 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:18:00.209709 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:00.209669 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:18:00.209936 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:00.209740 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:18:01.209609 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:01.209571 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:18:01.209911 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:01.209869 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:18:02.209770 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:02.209724 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" start-of-body= Nov 04 02:18:02.210163 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:02.209808 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" podUID="3d139de1-dfac-49f1-a95d-4f2470c81a89" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.48:15021/healthz/ready\": dial tcp 10.133.0.48:15021: connect: connection refused" Nov 04 02:18:03.916233 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:03.916188 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:18:03.916690 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:03.916542 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:18:03.917461 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:03.917440 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-8rt47" Nov 04 02:18:04.503783 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:18:04.503744 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:18:10.439723 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:10.439681 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:18:10.440209 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:10.439892 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" containerID="cri-o://720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54" gracePeriod=30 Nov 04 02:18:10.456840 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:10.456808 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:18:10.457046 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:10.457020 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" containerID="cri-o://ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c" gracePeriod=30 Nov 04 02:18:15.720739 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.720713 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:18:15.754142 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.754116 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:18:15.802878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.802845 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.802905 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.802957 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.802978 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803008 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803070 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803037 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803084 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803121 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803159 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803207 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803242 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803296 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803344 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803432 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803445 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803442 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803575 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803622 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803672 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\") pod \"ffb697dc-59d8-4adc-90f9-5528919587ba\" (UID: \"ffb697dc-59d8-4adc-90f9-5528919587ba\") " Nov 04 02:18:15.803803 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.803712 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\") pod \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\" (UID: \"3b25176b-0f1c-4cae-ad1c-ee9699d253fd\") " Nov 04 02:18:15.804126 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804045 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.804796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804256 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.804796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804361 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.804796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804383 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.804796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804396 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.804796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804724 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.805153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804806 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.805153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804853 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:18:15.805153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804913 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.805153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804913 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:18:15.805153 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.804972 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.806109 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806081 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "kube-api-access-hn7ch". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:18:15.806333 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806309 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.806645 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806618 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:18:15.806752 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806651 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "kube-api-access-4r2hk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:18:15.806752 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806672 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:18:15.806752 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.806745 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy" pod "3b25176b-0f1c-4cae-ad1c-ee9699d253fd" (UID: "3b25176b-0f1c-4cae-ad1c-ee9699d253fd"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:18:15.807062 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.807041 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:18:15.807404 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.807388 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token" pod "ffb697dc-59d8-4adc-90f9-5528919587ba" (UID: "ffb697dc-59d8-4adc-90f9-5528919587ba"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:18:15.905946 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.905917 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4r2hk\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-kube-api-access-4r2hk\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.905946 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.905946 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.905958 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.905985 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/ffb697dc-59d8-4adc-90f9-5528919587ba-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.905995 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906005 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906013 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906022 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hn7ch\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-kube-api-access-hn7ch\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906031 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/ffb697dc-59d8-4adc-90f9-5528919587ba-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906039 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906048 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906055 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906063 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906070 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:15.906144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:15.906078 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3b25176b-0f1c-4cae-ad1c-ee9699d253fd-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:18:16.400247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.400212 5029 generic.go:356] "Generic (PLEG): container finished" podID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerID="720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54" exitCode=0 Nov 04 02:18:16.400441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.400277 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" Nov 04 02:18:16.400441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.400300 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" event={"ID":"3b25176b-0f1c-4cae-ad1c-ee9699d253fd","Type":"ContainerDied","Data":"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54"} Nov 04 02:18:16.400441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.400378 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44" event={"ID":"3b25176b-0f1c-4cae-ad1c-ee9699d253fd","Type":"ContainerDied","Data":"9528bb005fa0dc1cb9e2973f2d531590974bc557b4e6bf878d9e12ee1a14dabe"} Nov 04 02:18:16.400441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.400401 5029 scope.go:122] "RemoveContainer" containerID="720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54" Nov 04 02:18:16.401746 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.401721 5029 generic.go:356] "Generic (PLEG): container finished" podID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerID="ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c" exitCode=0 Nov 04 02:18:16.401836 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.401784 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" Nov 04 02:18:16.401892 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.401787 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" event={"ID":"ffb697dc-59d8-4adc-90f9-5528919587ba","Type":"ContainerDied","Data":"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c"} Nov 04 02:18:16.401952 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.401926 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp" event={"ID":"ffb697dc-59d8-4adc-90f9-5528919587ba","Type":"ContainerDied","Data":"2a61a53abd3115434536a375410874835f7489ea5341a6718adbef853c35c6f1"} Nov 04 02:18:16.411686 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.411637 5029 scope.go:122] "RemoveContainer" containerID="720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54" Nov 04 02:18:16.411994 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:18:16.411969 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54\": container with ID starting with 720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54 not found: ID does not exist" containerID="720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54" Nov 04 02:18:16.412102 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.412001 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54"} err="failed to get container status \"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54\": rpc error: code = NotFound desc = could not find container \"720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54\": container with ID starting with 720286a0774898837fdfd7dc4cf651d47dc3c7f22f634bb4beaf0d82d9739b54 not found: ID does not exist" Nov 04 02:18:16.412102 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.412026 5029 scope.go:122] "RemoveContainer" containerID="ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c" Nov 04 02:18:16.420920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.420899 5029 scope.go:122] "RemoveContainer" containerID="ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c" Nov 04 02:18:16.421232 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:18:16.421214 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c\": container with ID starting with ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c not found: ID does not exist" containerID="ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c" Nov 04 02:18:16.421286 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.421241 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c"} err="failed to get container status \"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c\": rpc error: code = NotFound desc = could not find container \"ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c\": container with ID starting with ab69438b595c1db307c4b37945248d4963fdfb8d5b09e7cb4c0ded25c232ba7c not found: ID does not exist" Nov 04 02:18:16.565276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.565233 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:18:16.614589 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.614553 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-8wt44"] Nov 04 02:18:16.678553 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.678436 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:18:16.684868 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:16.684837 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w1-90ad0f-openshift-default-588c897dc9-m82rp"] Nov 04 02:18:17.621203 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:17.621166 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" path="/var/lib/kubelet/pods/3b25176b-0f1c-4cae-ad1c-ee9699d253fd/volumes" Nov 04 02:18:17.621821 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:17.621804 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" path="/var/lib/kubelet/pods/ffb697dc-59d8-4adc-90f9-5528919587ba/volumes" Nov 04 02:18:34.502571 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:18:34.502532 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:18:34.510854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.510824 5029 generic.go:356] "Generic (PLEG): container finished" podID="7730d765-c285-4766-97be-35765e0ea2c2" containerID="98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47" exitCode=0 Nov 04 02:18:34.511007 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.510893 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerDied","Data":"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47"} Nov 04 02:18:34.511298 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.511280 5029 scope.go:122] "RemoveContainer" containerID="98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47" Nov 04 02:18:34.512730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.512702 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:18:34.512845 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.512784 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:18:34.512845 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:34.512834 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:18:35.517770 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:35.517731 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerStarted","Data":"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19"} Nov 04 02:18:36.172852 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:36.172818 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:18:36.173917 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:36.173891 5029 patch_prober.go:29] interesting pod/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:18:36.174040 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:36.173942 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:18:37.173590 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:37.173555 5029 patch_prober.go:29] interesting pod/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:18:37.173992 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:37.173626 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:18:38.173494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:38.173447 5029 patch_prober.go:29] interesting pod/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:18:38.173705 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:38.173510 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:18:39.177587 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:39.173181 5029 patch_prober.go:29] interesting pod/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:18:39.177587 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:39.173257 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:18:39.965106 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:39.965068 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:18:40.175979 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:40.175939 5029 patch_prober.go:29] interesting pod/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" start-of-body= Nov 04 02:18:40.176302 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:40.176239 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.46:15021/healthz/ready\": dial tcp 10.133.0.46:15021: connect: connection refused" Nov 04 02:18:41.177538 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:41.177506 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:18:41.178666 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:18:41.178646 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:19:04.503128 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:19:04.503039 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:19:14.388164 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:14.388129 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:19:14.439122 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:14.439092 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:19:29.408238 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.408197 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" podUID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" containerName="main" containerID="cri-o://30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570" gracePeriod=45 Nov 04 02:19:29.459175 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.459126 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" podUID="dc4659f1-1866-4e10-84dc-e6942a2142f5" containerName="main" containerID="cri-o://52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc" gracePeriod=45 Nov 04 02:19:29.718758 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.718734 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:19:29.741711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.741691 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:19:29.786475 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.786446 5029 generic.go:356] "Generic (PLEG): container finished" podID="dc4659f1-1866-4e10-84dc-e6942a2142f5" containerID="52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc" exitCode=0 Nov 04 02:19:29.786604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.786503 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" Nov 04 02:19:29.786604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.786542 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" event={"ID":"dc4659f1-1866-4e10-84dc-e6942a2142f5","Type":"ContainerDied","Data":"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc"} Nov 04 02:19:29.786604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.786583 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx" event={"ID":"dc4659f1-1866-4e10-84dc-e6942a2142f5","Type":"ContainerDied","Data":"34ca4d484a7297f1f2e3061be0e7f8f945de2c07a74ea3cfd7464bf084143c2c"} Nov 04 02:19:29.786761 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.786607 5029 scope.go:122] "RemoveContainer" containerID="52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc" Nov 04 02:19:29.788007 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.787953 5029 generic.go:356] "Generic (PLEG): container finished" podID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" containerID="30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570" exitCode=0 Nov 04 02:19:29.788007 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.787994 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" Nov 04 02:19:29.788188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.788032 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" event={"ID":"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6","Type":"ContainerDied","Data":"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570"} Nov 04 02:19:29.788188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.788059 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq" event={"ID":"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6","Type":"ContainerDied","Data":"967019aa98ae0b8f9f6186e5b194007956755e7c66e76ed70488ac44457a3776"} Nov 04 02:19:29.794883 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.794867 5029 scope.go:122] "RemoveContainer" containerID="52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc" Nov 04 02:19:29.795139 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:19:29.795115 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc\": container with ID starting with 52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc not found: ID does not exist" containerID="52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc" Nov 04 02:19:29.795202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.795148 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc"} err="failed to get container status \"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc\": rpc error: code = NotFound desc = could not find container \"52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc\": container with ID starting with 52bd0f0613b698016c455fe22272500937c41a6e9b8e78cceba2fffecf7b2adc not found: ID does not exist" Nov 04 02:19:29.795202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.795163 5029 scope.go:122] "RemoveContainer" containerID="30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570" Nov 04 02:19:29.802862 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.802842 5029 scope.go:122] "RemoveContainer" containerID="30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570" Nov 04 02:19:29.803130 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:19:29.803109 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570\": container with ID starting with 30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570 not found: ID does not exist" containerID="30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570" Nov 04 02:19:29.803205 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.803135 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570"} err="failed to get container status \"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570\": rpc error: code = NotFound desc = could not find container \"30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570\": container with ID starting with 30388b09f261e46756167c4094708a8fa4cf05fd8c0f709cc090e3786560e570 not found: ID does not exist" Nov 04 02:19:29.825460 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825437 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.825579 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825485 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.825579 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825515 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.825651 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825618 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\" (UniqueName: \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.825690 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825670 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.825754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825732 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.825754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825747 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.825931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825784 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.825931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825870 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.825931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825903 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.826095 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825955 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\" (UniqueName: \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.826095 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.825978 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.826095 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826018 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\") pod \"dc4659f1-1866-4e10-84dc-e6942a2142f5\" (UID: \"dc4659f1-1866-4e10-84dc-e6942a2142f5\") " Nov 04 02:19:29.826257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826084 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\" (UniqueName: \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\") pod \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\" (UID: \"fa4e95d2-a4e3-4813-8e01-25a0d07cebf6\") " Nov 04 02:19:29.826257 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826215 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.826390 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826369 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.826784 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826600 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.826784 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826619 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826784 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826803 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826818 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826832 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826845 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.826955 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.826858 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.828174 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828155 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.828302 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828284 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "kube-api-access-7sx9z". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:19:29.828443 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828426 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs" pod "dc4659f1-1866-4e10-84dc-e6942a2142f5" (UID: "dc4659f1-1866-4e10-84dc-e6942a2142f5"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:19:29.828565 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828541 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:19:29.828769 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828749 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:19:29.828870 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.828846 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f" pod "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" (UID: "fa4e95d2-a4e3-4813-8e01-25a0d07cebf6"). InnerVolumeSpecName "kube-api-access-jt47f". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:19:29.928166 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928133 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.928166 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928164 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jt47f\" (UniqueName: \"kubernetes.io/projected/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-kube-api-access-jt47f\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.928166 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928174 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.928392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928184 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/dc4659f1-1866-4e10-84dc-e6942a2142f5-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.928392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928193 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/dc4659f1-1866-4e10-84dc-e6942a2142f5-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:29.928392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:29.928201 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-7sx9z\" (UniqueName: \"kubernetes.io/projected/dc4659f1-1866-4e10-84dc-e6942a2142f5-kube-api-access-7sx9z\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:19:30.209650 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:30.209620 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:19:30.214944 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:30.214915 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/simulator-w3-3bf869-kserve-54bd6944b8-7xfbx"] Nov 04 02:19:30.323093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:30.323059 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:19:30.329391 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:30.329368 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w3-3bf869/premium-w3-3bf869-kserve-5f6b8fc7-wzqrq"] Nov 04 02:19:31.619139 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:31.619107 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="dc4659f1-1866-4e10-84dc-e6942a2142f5" path="/var/lib/kubelet/pods/dc4659f1-1866-4e10-84dc-e6942a2142f5/volumes" Nov 04 02:19:31.619538 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:19:31.619505 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" path="/var/lib/kubelet/pods/fa4e95d2-a4e3-4813-8e01-25a0d07cebf6/volumes" Nov 04 02:19:34.502201 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:19:34.502157 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:20:04.502936 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:20:04.502897 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:20:34.501532 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:20:34.501492 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:20:47.838586 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:47.838550 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:20:47.839079 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:47.838762 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" podUID="7d74fa53-825d-4734-b4db-4dc70459cd6a" containerName="maas-api" containerID="cri-o://20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0" gracePeriod=30 Nov 04 02:20:48.126680 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.126658 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:20:48.167719 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.167686 5029 generic.go:356] "Generic (PLEG): container finished" podID="7d74fa53-825d-4734-b4db-4dc70459cd6a" containerID="20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0" exitCode=0 Nov 04 02:20:48.167878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.167739 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" Nov 04 02:20:48.167878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.167774 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" event={"ID":"7d74fa53-825d-4734-b4db-4dc70459cd6a","Type":"ContainerDied","Data":"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0"} Nov 04 02:20:48.167878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.167840 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v" event={"ID":"7d74fa53-825d-4734-b4db-4dc70459cd6a","Type":"ContainerDied","Data":"cf030fb572ba298205cc43ce1544d90aeb0dcec138d0f8994d872f2109439ccf"} Nov 04 02:20:48.167878 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.167870 5029 scope.go:122] "RemoveContainer" containerID="20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0" Nov 04 02:20:48.178308 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.178286 5029 scope.go:122] "RemoveContainer" containerID="20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0" Nov 04 02:20:48.178776 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:20:48.178753 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0\": container with ID starting with 20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0 not found: ID does not exist" containerID="20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0" Nov 04 02:20:48.178887 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.178785 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0"} err="failed to get container status \"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0\": rpc error: code = NotFound desc = could not find container \"20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0\": container with ID starting with 20afd2f4542e4578a4d417963ec902ad11d78539cb16138b560225c0328720e0 not found: ID does not exist" Nov 04 02:20:48.188184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188159 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\") pod \"7d74fa53-825d-4734-b4db-4dc70459cd6a\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " Nov 04 02:20:48.188289 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188278 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\") pod \"7d74fa53-825d-4734-b4db-4dc70459cd6a\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " Nov 04 02:20:48.188410 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188393 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\" (UniqueName: \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\") pod \"7d74fa53-825d-4734-b4db-4dc70459cd6a\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " Nov 04 02:20:48.188522 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188507 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") pod \"7d74fa53-825d-4734-b4db-4dc70459cd6a\" (UID: \"7d74fa53-825d-4734-b4db-4dc70459cd6a\") " Nov 04 02:20:48.188684 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188659 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca" pod "7d74fa53-825d-4734-b4db-4dc70459cd6a" (UID: "7d74fa53-825d-4734-b4db-4dc70459cd6a"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:20:48.188969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.188948 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/7d74fa53-825d-4734-b4db-4dc70459cd6a-openshift-service-ca\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:48.190988 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.190966 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs" pod "7d74fa53-825d-4734-b4db-4dc70459cd6a" (UID: "7d74fa53-825d-4734-b4db-4dc70459cd6a"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:20:48.191093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.191014 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks" pod "7d74fa53-825d-4734-b4db-4dc70459cd6a" (UID: "7d74fa53-825d-4734-b4db-4dc70459cd6a"). InnerVolumeSpecName "kube-api-access-qdhks". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:20:48.191154 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.191144 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls" pod "7d74fa53-825d-4734-b4db-4dc70459cd6a" (UID: "7d74fa53-825d-4734-b4db-4dc70459cd6a"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:20:48.290028 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.289957 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qdhks\" (UniqueName: \"kubernetes.io/projected/7d74fa53-825d-4734-b4db-4dc70459cd6a-kube-api-access-qdhks\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:48.290028 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.289998 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-maas-api-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:48.290028 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.290010 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/7d74fa53-825d-4734-b4db-4dc70459cd6a-metrics-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:20:48.587545 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.587509 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:20:48.594453 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:48.594428 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-extmod-a-0508d1-7b7bb4975c-lvm2v"] Nov 04 02:20:49.619082 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:20:49.619046 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7d74fa53-825d-4734-b4db-4dc70459cd6a" path="/var/lib/kubelet/pods/7d74fa53-825d-4734-b4db-4dc70459cd6a/volumes" Nov 04 02:21:04.502553 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:04.502514 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:21:34.501519 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:34.501462 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:21:35.448190 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:35.448156 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:21:35.448408 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:35.448385 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" containerID="cri-o://524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19" gracePeriod=30 Nov 04 02:21:36.783343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:36.783302 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:21:36.783932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:36.783569 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" containerID="cri-o://a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d" gracePeriod=30 Nov 04 02:21:37.339959 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:37.339920 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:21:37.340597 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:37.340560 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" containerID="cri-o://5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92" gracePeriod=30 Nov 04 02:21:40.701951 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.701924 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:21:40.809479 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809452 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809520 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809602 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809572 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809600 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809627 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809658 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809689 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809909 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809747 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809909 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809685 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:40.809909 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809814 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\") pod \"7730d765-c285-4766-97be-35765e0ea2c2\" (UID: \"7730d765-c285-4766-97be-35765e0ea2c2\") " Nov 04 02:21:40.809909 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809792 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:40.810118 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.809953 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:40.810118 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810021 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:40.810118 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810049 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:21:40.810276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810170 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.810276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810190 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/7730d765-c285-4766-97be-35765e0ea2c2-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.810276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810204 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.810276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810219 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.810276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.810229 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.811947 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.811921 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:40.811947 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.811931 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:21:40.812085 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.811924 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:40.812213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.812190 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5" pod "7730d765-c285-4766-97be-35765e0ea2c2" (UID: "7730d765-c285-4766-97be-35765e0ea2c2"). InnerVolumeSpecName "kube-api-access-qbnd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:40.910943 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.910912 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/7730d765-c285-4766-97be-35765e0ea2c2-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.910943 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.910938 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.910943 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.910948 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/7730d765-c285-4766-97be-35765e0ea2c2-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.911157 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.910957 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qbnd5\" (UniqueName: \"kubernetes.io/projected/7730d765-c285-4766-97be-35765e0ea2c2-kube-api-access-qbnd5\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:40.927622 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:40.927598 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:21:41.436530 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.436502 5029 generic.go:356] "Generic (PLEG): container finished" podID="7730d765-c285-4766-97be-35765e0ea2c2" containerID="524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19" exitCode=0 Nov 04 02:21:41.436722 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.436550 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" Nov 04 02:21:41.436722 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.436554 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerDied","Data":"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19"} Nov 04 02:21:41.436722 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.436656 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9" event={"ID":"7730d765-c285-4766-97be-35765e0ea2c2","Type":"ContainerDied","Data":"236b949e743c75eb98840ff40d85482d9504b1d15a5d3c1be6306f08a9168899"} Nov 04 02:21:41.436722 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.436678 5029 scope.go:122] "RemoveContainer" containerID="524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19" Nov 04 02:21:41.446579 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.446559 5029 scope.go:122] "RemoveContainer" containerID="98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47" Nov 04 02:21:41.454680 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.454662 5029 scope.go:122] "RemoveContainer" containerID="524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19" Nov 04 02:21:41.454938 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:41.454919 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19\": container with ID starting with 524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19 not found: ID does not exist" containerID="524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19" Nov 04 02:21:41.454995 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.454945 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19"} err="failed to get container status \"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19\": rpc error: code = NotFound desc = could not find container \"524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19\": container with ID starting with 524d0d193c4ac0264515fa73272ac6d1cdc53421418ef8169743562ab2538f19 not found: ID does not exist" Nov 04 02:21:41.454995 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.454963 5029 scope.go:122] "RemoveContainer" containerID="98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47" Nov 04 02:21:41.455202 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:41.455184 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47\": container with ID starting with 98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47 not found: ID does not exist" containerID="98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47" Nov 04 02:21:41.455239 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.455210 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47"} err="failed to get container status \"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47\": rpc error: code = NotFound desc = could not find container \"98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47\": container with ID starting with 98cb6cac12bc29698861e992f9a330c836b7f97881a3a4e3d900dcf4ffc2be47 not found: ID does not exist" Nov 04 02:21:41.603971 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.603947 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:21:41.608020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.608002 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-extmod-b-3f76d0-openshift-default-6f4f58d5fc-vgtt9"] Nov 04 02:21:41.619655 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:41.619633 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="7730d765-c285-4766-97be-35765e0ea2c2" path="/var/lib/kubelet/pods/7730d765-c285-4766-97be-35765e0ea2c2/volumes" Nov 04 02:21:42.044175 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.044108 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:21:42.122118 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122084 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122312 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122150 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122312 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122200 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122312 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122231 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122312 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122259 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122342 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122383 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122428 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122458 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\") pod \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\" (UID: \"f84592a2-efda-4807-b2a5-57a6a9d0b48f\") " Nov 04 02:21:42.122673 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122556 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.123136 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.122922 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.123136 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.123102 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.123414 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.123367 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.123414 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.123375 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:21:42.123635 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.123606 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.125566 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.125531 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "kube-api-access-fl2vx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:42.125566 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.125541 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:42.125779 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.125753 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:21:42.125887 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.125825 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy" pod "f84592a2-efda-4807-b2a5-57a6a9d0b48f" (UID: "f84592a2-efda-4807-b2a5-57a6a9d0b48f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.223568 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223527 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223568 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223556 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223568 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223564 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223568 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223574 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fl2vx\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-kube-api-access-fl2vx\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223923 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223584 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223923 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223592 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223923 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223600 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.223923 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.223609 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f84592a2-efda-4807-b2a5-57a6a9d0b48f-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.444582 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.444542 5029 generic.go:356] "Generic (PLEG): container finished" podID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerID="a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d" exitCode=0 Nov 04 02:21:42.444756 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.444610 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" Nov 04 02:21:42.444756 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.444627 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" event={"ID":"f84592a2-efda-4807-b2a5-57a6a9d0b48f","Type":"ContainerDied","Data":"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d"} Nov 04 02:21:42.444756 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.444702 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55" event={"ID":"f84592a2-efda-4807-b2a5-57a6a9d0b48f","Type":"ContainerDied","Data":"f2bb8e9c81bd670b5dbfd63e7c45c6857c517196d3113b6da2a0e84c7215334c"} Nov 04 02:21:42.444756 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.444735 5029 scope.go:122] "RemoveContainer" containerID="a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d" Nov 04 02:21:42.462730 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.462706 5029 scope.go:122] "RemoveContainer" containerID="a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d" Nov 04 02:21:42.463081 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:42.462989 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d\": container with ID starting with a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d not found: ID does not exist" containerID="a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d" Nov 04 02:21:42.463081 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.463025 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d"} err="failed to get container status \"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d\": rpc error: code = NotFound desc = could not find container \"a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d\": container with ID starting with a82b138d2f1ba31d168cc409d971924a908d8f9c0b3ab37e0682d1e208aa039d not found: ID does not exist" Nov 04 02:21:42.611024 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.610993 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:21:42.614303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.614277 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-hpj55"] Nov 04 02:21:42.619188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.619166 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:21:42.729796 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.729764 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.729983 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.729823 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.729983 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.729857 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.729983 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.729908 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.729983 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.729949 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.730195 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730006 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.730195 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730055 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.730195 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730106 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.730195 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730161 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.730400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730199 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\") pod \"434d0892-a3a8-44a8-a187-765bbd920700\" (UID: \"434d0892-a3a8-44a8-a187-765bbd920700\") " Nov 04 02:21:42.730400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730320 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.730650 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730625 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.730650 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730636 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:21:42.730781 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730655 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.730907 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730886 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.731140 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.730946 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.733059 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.733023 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:42.733201 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.733170 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:21:42.733201 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.733179 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:42.733384 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.733282 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq" pod "434d0892-a3a8-44a8-a187-765bbd920700" (UID: "434d0892-a3a8-44a8-a187-765bbd920700"). InnerVolumeSpecName "kube-api-access-ldgtq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:42.832066 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832031 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832066 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832069 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832085 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832099 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/434d0892-a3a8-44a8-a187-765bbd920700-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832114 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ldgtq\" (UniqueName: \"kubernetes.io/projected/434d0892-a3a8-44a8-a187-765bbd920700-kube-api-access-ldgtq\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832128 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/434d0892-a3a8-44a8-a187-765bbd920700-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:42.832285 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:42.832149 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/434d0892-a3a8-44a8-a187-765bbd920700-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:43.476496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.474875 5029 generic.go:356] "Generic (PLEG): container finished" podID="434d0892-a3a8-44a8-a187-765bbd920700" containerID="5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92" exitCode=0 Nov 04 02:21:43.476496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.474991 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" event={"ID":"434d0892-a3a8-44a8-a187-765bbd920700","Type":"ContainerDied","Data":"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92"} Nov 04 02:21:43.476496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.475072 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" event={"ID":"434d0892-a3a8-44a8-a187-765bbd920700","Type":"ContainerDied","Data":"2549cc8c97a81dc9bc1e25964c2dc1bb7cfc019263aa4beb3344fc49bea20f02"} Nov 04 02:21:43.476496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.475102 5029 scope.go:122] "RemoveContainer" containerID="5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92" Nov 04 02:21:43.476496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.475244 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj" Nov 04 02:21:43.520595 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.520567 5029 scope.go:122] "RemoveContainer" containerID="5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92" Nov 04 02:21:43.520960 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:43.520932 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92\": container with ID starting with 5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92 not found: ID does not exist" containerID="5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92" Nov 04 02:21:43.521002 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.520975 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92"} err="failed to get container status \"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92\": rpc error: code = NotFound desc = could not find container \"5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92\": container with ID starting with 5cf5bd0bb5dfbe6615e0001ae038d494aa6730b2d8d5d0c2a6a7211cb3fc6f92 not found: ID does not exist" Nov 04 02:21:43.621891 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.621863 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" path="/var/lib/kubelet/pods/f84592a2-efda-4807-b2a5-57a6a9d0b48f/volumes" Nov 04 02:21:43.654493 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.645454 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:21:43.696494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:43.695485 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-fwbsj"] Nov 04 02:21:45.620315 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:45.620281 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="434d0892-a3a8-44a8-a187-765bbd920700" path="/var/lib/kubelet/pods/434d0892-a3a8-44a8-a187-765bbd920700/volumes" Nov 04 02:21:55.947142 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:55.947074 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerName="main" containerID="cri-o://d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae" gracePeriod=45 Nov 04 02:21:55.964747 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:55.964716 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerName="main" probeResult="failure" output="Get \"https://10.133.0.31:8000/ready\": dial tcp 10.133.0.31:8000: connect: connection refused" Nov 04 02:21:56.193273 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.193245 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:21:56.282393 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282364 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\" (UniqueName: \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.282604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282416 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\" (UniqueName: \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.282604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282530 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.282604 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282590 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.282788 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282686 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.282788 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.282738 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\") pod \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\" (UID: \"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3\") " Nov 04 02:21:56.283081 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.283031 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:56.283081 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.283037 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:56.283081 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.283061 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:56.284890 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.284863 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:21:56.285001 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.284905 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:21:56.285001 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.284970 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk" pod "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" (UID: "29a11f02-1e58-407e-a7b5-5dbbdd48d4a3"). InnerVolumeSpecName "kube-api-access-db2vk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:21:56.383551 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383515 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.383551 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383548 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.383551 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383558 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.383819 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383568 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-db2vk\" (UniqueName: \"kubernetes.io/projected/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-kube-api-access-db2vk\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.383819 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383577 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.383819 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.383585 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:21:56.548932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.548849 5029 generic.go:356] "Generic (PLEG): container finished" podID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerID="d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae" exitCode=0 Nov 04 02:21:56.548932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.548898 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" event={"ID":"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3","Type":"ContainerDied","Data":"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae"} Nov 04 02:21:56.548932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.548900 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" Nov 04 02:21:56.548932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.548929 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd" event={"ID":"29a11f02-1e58-407e-a7b5-5dbbdd48d4a3","Type":"ContainerDied","Data":"925290135b9790b5c61ab71e4fe9269fd2abc49adb5be48246a10ec763ab70c8"} Nov 04 02:21:56.549232 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.548946 5029 scope.go:122] "RemoveContainer" containerID="d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae" Nov 04 02:21:56.558141 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.558122 5029 scope.go:122] "RemoveContainer" containerID="d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae" Nov 04 02:21:56.558421 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:21:56.558393 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae\": container with ID starting with d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae not found: ID does not exist" containerID="d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae" Nov 04 02:21:56.558485 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.558431 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae"} err="failed to get container status \"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae\": rpc error: code = NotFound desc = could not find container \"d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae\": container with ID starting with d5a6ad8534bd0dc76f1d4e3264ce9ccb99c5e3c0f264b973c42658edf7a6ecae not found: ID does not exist" Nov 04 02:21:56.712335 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.712304 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:21:56.716039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:56.716014 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w0-24bea9/simulator-w0-24bea9-kserve-5669f5fd56-szbpd"] Nov 04 02:21:57.619936 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:21:57.619899 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" path="/var/lib/kubelet/pods/29a11f02-1e58-407e-a7b5-5dbbdd48d4a3/volumes" Nov 04 02:22:04.338637 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:04.338596 5029 patch_prober.go:29] interesting pod/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj container/istio-proxy namespace/openshift-ingress: Readiness probe status=failure output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" start-of-body= Nov 04 02:22:04.339090 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:04.338667 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="openshift-ingress/maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj" podUID="f42723eb-4c4b-490d-82d0-4fa690ee2a8a" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.47:15021/healthz/ready\": context deadline exceeded (Client.Timeout exceeded while awaiting headers)" Nov 04 02:22:04.501920 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:04.501882 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:22:06.478531 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:06.478438 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:22:06.479006 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:06.478717 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" containerID="cri-o://41e5015bbc9c9de34d2929796aa0900ee3a6e46171c620e5d33ce38bb075b6b8" gracePeriod=30 Nov 04 02:22:10.833715 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:10.833686 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:22:10.834123 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:10.833875 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" podUID="07aa0d55-0d45-4a69-ab12-105745a1e275" containerName="maas-api" containerID="cri-o://4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742" gracePeriod=30 Nov 04 02:22:11.092425 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.092359 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:22:11.166586 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.166560 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\") pod \"07aa0d55-0d45-4a69-ab12-105745a1e275\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " Nov 04 02:22:11.166761 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.166594 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") pod \"07aa0d55-0d45-4a69-ab12-105745a1e275\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " Nov 04 02:22:11.166761 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.166681 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\") pod \"07aa0d55-0d45-4a69-ab12-105745a1e275\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " Nov 04 02:22:11.166761 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.166714 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\" (UniqueName: \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\") pod \"07aa0d55-0d45-4a69-ab12-105745a1e275\" (UID: \"07aa0d55-0d45-4a69-ab12-105745a1e275\") " Nov 04 02:22:11.167043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.167016 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca" pod "07aa0d55-0d45-4a69-ab12-105745a1e275" (UID: "07aa0d55-0d45-4a69-ab12-105745a1e275"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:11.168844 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.168818 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq" pod "07aa0d55-0d45-4a69-ab12-105745a1e275" (UID: "07aa0d55-0d45-4a69-ab12-105745a1e275"). InnerVolumeSpecName "kube-api-access-r6txq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:11.168844 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.168831 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls" pod "07aa0d55-0d45-4a69-ab12-105745a1e275" (UID: "07aa0d55-0d45-4a69-ab12-105745a1e275"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:11.168998 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.168887 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs" pod "07aa0d55-0d45-4a69-ab12-105745a1e275" (UID: "07aa0d55-0d45-4a69-ab12-105745a1e275"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:11.268084 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.268052 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-metrics-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.268084 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.268079 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/07aa0d55-0d45-4a69-ab12-105745a1e275-maas-api-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.268084 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.268089 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/07aa0d55-0d45-4a69-ab12-105745a1e275-openshift-service-ca\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.268375 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.268098 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r6txq\" (UniqueName: \"kubernetes.io/projected/07aa0d55-0d45-4a69-ab12-105745a1e275-kube-api-access-r6txq\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.645015 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.644984 5029 generic.go:356] "Generic (PLEG): container finished" podID="418ac499-1567-4c0d-a745-778881063cb2" containerID="41e5015bbc9c9de34d2929796aa0900ee3a6e46171c620e5d33ce38bb075b6b8" exitCode=0 Nov 04 02:22:11.645179 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.645056 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" event={"ID":"418ac499-1567-4c0d-a745-778881063cb2","Type":"ContainerDied","Data":"41e5015bbc9c9de34d2929796aa0900ee3a6e46171c620e5d33ce38bb075b6b8"} Nov 04 02:22:11.646526 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.646499 5029 generic.go:356] "Generic (PLEG): container finished" podID="07aa0d55-0d45-4a69-ab12-105745a1e275" containerID="4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742" exitCode=0 Nov 04 02:22:11.646654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.646534 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" Nov 04 02:22:11.646654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.646562 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" event={"ID":"07aa0d55-0d45-4a69-ab12-105745a1e275","Type":"ContainerDied","Data":"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742"} Nov 04 02:22:11.646654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.646616 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd" event={"ID":"07aa0d55-0d45-4a69-ab12-105745a1e275","Type":"ContainerDied","Data":"7873e72b044d17d60895cbdf74e884edddd99e3980e2ea13f0c964a0725fb042"} Nov 04 02:22:11.646654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.646647 5029 scope.go:122] "RemoveContainer" containerID="4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742" Nov 04 02:22:11.658739 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.658680 5029 scope.go:122] "RemoveContainer" containerID="4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742" Nov 04 02:22:11.659049 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:11.659022 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742\": container with ID starting with 4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742 not found: ID does not exist" containerID="4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742" Nov 04 02:22:11.659146 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.659061 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742"} err="failed to get container status \"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742\": rpc error: code = NotFound desc = could not find container \"4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742\": container with ID starting with 4111ec2ab60d84b244bad451e9040f9b779bf8eda5febb89cc9e899cfbf63742 not found: ID does not exist" Nov 04 02:22:11.715317 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.715296 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:22:11.773942 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.773886 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.773942 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.773921 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.773959 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.773990 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774024 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774057 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774125 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774172 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774187 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774189 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\") pod \"418ac499-1567-4c0d-a745-778881063cb2\" (UID: \"418ac499-1567-4c0d-a745-778881063cb2\") " Nov 04 02:22:11.774565 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774189 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:11.774565 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774407 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:11.774565 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774491 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:11.774723 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774596 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.774723 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774615 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.774723 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774618 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:11.774723 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.774631 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.775388 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.775358 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:11.776954 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.776637 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:11.776954 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.776911 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:11.777240 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.777214 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:11.777495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.777449 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv" pod "418ac499-1567-4c0d-a745-778881063cb2" (UID: "418ac499-1567-4c0d-a745-778881063cb2"). InnerVolumeSpecName "kube-api-access-wm8dv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:11.810143 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.810114 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:22:11.814045 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.814023 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w0-24bea9-5ff6d5f97f-phsxd"] Nov 04 02:22:11.875309 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875281 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wm8dv\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-kube-api-access-wm8dv\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.875309 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875309 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/418ac499-1567-4c0d-a745-778881063cb2-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.875671 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875324 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.875671 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875337 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/418ac499-1567-4c0d-a745-778881063cb2-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.875671 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875350 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/418ac499-1567-4c0d-a745-778881063cb2-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:11.875671 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:11.875363 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/418ac499-1567-4c0d-a745-778881063cb2-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:12.652632 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:12.652600 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" Nov 04 02:22:12.652632 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:12.652607 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65" event={"ID":"418ac499-1567-4c0d-a745-778881063cb2","Type":"ContainerDied","Data":"df18413e5b764163c50fdabca77a52cb54485544bdeeefa8f6aac02dd635d874"} Nov 04 02:22:12.652888 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:12.652669 5029 scope.go:122] "RemoveContainer" containerID="41e5015bbc9c9de34d2929796aa0900ee3a6e46171c620e5d33ce38bb075b6b8" Nov 04 02:22:12.816623 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:12.816589 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:22:12.866538 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:12.866506 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-clb65"] Nov 04 02:22:13.624280 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:13.620183 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="07aa0d55-0d45-4a69-ab12-105745a1e275" path="/var/lib/kubelet/pods/07aa0d55-0d45-4a69-ab12-105745a1e275/volumes" Nov 04 02:22:13.624280 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:13.620890 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="418ac499-1567-4c0d-a745-778881063cb2" path="/var/lib/kubelet/pods/418ac499-1567-4c0d-a745-778881063cb2/volumes" Nov 04 02:22:15.746525 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:15.746496 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:22:22.355794 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.355758 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm"] Nov 04 02:22:22.356268 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356245 5029 cpu_manager.go:414] "containerMap: removing container" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" Nov 04 02:22:22.356322 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356272 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" Nov 04 02:22:22.356322 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356288 5029 cpu_manager.go:414] "containerMap: removing container" podUID="07aa0d55-0d45-4a69-ab12-105745a1e275" containerName="maas-api" Nov 04 02:22:22.356322 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356296 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="07aa0d55-0d45-4a69-ab12-105745a1e275" containerName="maas-api" Nov 04 02:22:22.356322 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356309 5029 cpu_manager.go:414] "containerMap: removing container" podUID="7d74fa53-825d-4734-b4db-4dc70459cd6a" containerName="maas-api" Nov 04 02:22:22.356322 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356316 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7d74fa53-825d-4734-b4db-4dc70459cd6a" containerName="maas-api" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356326 5029 cpu_manager.go:414] "containerMap: removing container" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356333 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356350 5029 cpu_manager.go:414] "containerMap: removing container" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356358 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356377 5029 cpu_manager.go:414] "containerMap: removing container" podUID="dc4659f1-1866-4e10-84dc-e6942a2142f5" containerName="main" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356386 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="dc4659f1-1866-4e10-84dc-e6942a2142f5" containerName="main" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356397 5029 cpu_manager.go:414] "containerMap: removing container" podUID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" containerName="main" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356405 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" containerName="main" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356420 5029 cpu_manager.go:414] "containerMap: removing container" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356428 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356491 5029 cpu_manager.go:414] "containerMap: removing container" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356499 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356501 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356520 5029 cpu_manager.go:414] "containerMap: removing container" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356527 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356535 5029 cpu_manager.go:414] "containerMap: removing container" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerName="main" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356539 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerName="main" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356545 5029 cpu_manager.go:414] "containerMap: removing container" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356549 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356616 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="418ac499-1567-4c0d-a745-778881063cb2" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356625 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356632 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3b25176b-0f1c-4cae-ad1c-ee9699d253fd" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356636 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="29a11f02-1e58-407e-a7b5-5dbbdd48d4a3" containerName="main" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356641 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="ffb697dc-59d8-4adc-90f9-5528919587ba" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356646 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="434d0892-a3a8-44a8-a187-765bbd920700" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356651 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="07aa0d55-0d45-4a69-ab12-105745a1e275" containerName="maas-api" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356657 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7d74fa53-825d-4734-b4db-4dc70459cd6a" containerName="maas-api" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356662 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="f84592a2-efda-4807-b2a5-57a6a9d0b48f" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356668 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="7730d765-c285-4766-97be-35765e0ea2c2" containerName="istio-proxy" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356673 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="dc4659f1-1866-4e10-84dc-e6942a2142f5" containerName="main" Nov 04 02:22:22.356874 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.356678 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="fa4e95d2-a4e3-4813-8e01-25a0d07cebf6" containerName="main" Nov 04 02:22:22.361214 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.361197 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.413645 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.413621 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w0-24bea9-openshift-default-dockercfg-g5z8q\"" Nov 04 02:22:22.464873 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.464848 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm"] Nov 04 02:22:22.473783 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.473761 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.473910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.473789 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.473910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.473817 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.473910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.473834 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.474026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.473997 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l4frg\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.474062 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.474051 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.474119 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.474102 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.474168 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.474155 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.474206 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.474198 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575304 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575276 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575337 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575357 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575373 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575405 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575423 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575450 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4frg\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575507 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575506 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575553 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575780 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575812 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.575893 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.575836 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.576093 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.576062 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.576353 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.576329 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.577907 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.577884 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.578036 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.578016 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.653126 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.652670 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm"] Nov 04 02:22:22.653126 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:22.653011 5029 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[istio-token kube-api-access-l4frg], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" podUID="037031dc-e8e2-48e9-842b-e1e85e0606f4" Nov 04 02:22:22.679568 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:22.679537 5029 projected.go:196] Error preparing data for projected volume kube-api-access-l4frg for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:22.679742 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:22.679593 5029 projected.go:196] Error preparing data for projected volume istio-token for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:22.679742 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:22.679628 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg podName:037031dc-e8e2-48e9-842b-e1e85e0606f4 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:23.179607336 +0000 UTC m=+1575.672932472 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-l4frg" (UniqueName: "kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:22.679742 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:22.679647 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token podName:037031dc-e8e2-48e9-842b-e1e85e0606f4 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:23.17963622 +0000 UTC m=+1575.672961369 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "istio-token" (UniqueName: "kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:22.696463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.696432 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.707088 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.707062 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:22.778784 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.778750 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.778964 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.778827 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.778964 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.778912 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.779073 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.778982 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.779073 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779059 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.779184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779071 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:22.779184 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779151 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.779283 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779175 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:22:22.779283 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779228 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\") pod \"037031dc-e8e2-48e9-842b-e1e85e0606f4\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " Nov 04 02:22:22.779385 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779273 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:22.779495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779454 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:22.779495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779486 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:22.779665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779648 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.779718 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779672 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.779718 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779686 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.779718 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779701 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/037031dc-e8e2-48e9-842b-e1e85e0606f4-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.779718 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.779713 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.781440 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.781415 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:22:22.781863 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.781840 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy" pod "037031dc-e8e2-48e9-842b-e1e85e0606f4" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:22.880647 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.880621 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:22.880647 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:22.880644 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.183247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.183216 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:23.183432 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.183260 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l4frg\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg\") pod \"e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm\" (UID: \"037031dc-e8e2-48e9-842b-e1e85e0606f4\") " pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:23.235354 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:23.235329 5029 projected.go:196] Error preparing data for projected volume istio-token for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:23.235499 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:23.235386 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token podName:037031dc-e8e2-48e9-842b-e1e85e0606f4 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:24.23537196 +0000 UTC m=+1576.728697078 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "istio-token" (UniqueName: "kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:23.235499 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:23.235447 5029 projected.go:196] Error preparing data for projected volume kube-api-access-l4frg for pod openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm: failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:23.235617 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:23.235517 5029 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg podName:037031dc-e8e2-48e9-842b-e1e85e0606f4 nodeName:}" failed. No retries permitted until 2026-11-04 02:22:24.235504204 +0000 UTC m=+1576.728829334 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-l4frg" (UniqueName: "kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg") pod "e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" (UID: "037031dc-e8e2-48e9-842b-e1e85e0606f4") : failed to fetch token: serviceaccounts "e2e-worker-w0-24bea9-openshift-default" not found Nov 04 02:22:23.703667 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.701453 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm" Nov 04 02:22:23.932031 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.931959 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm"] Nov 04 02:22:23.936492 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.936444 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w0-24bea9-openshift-default-7f5bd59d8c-nvbnm"] Nov 04 02:22:23.992697 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.992671 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-l4frg\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-kube-api-access-l4frg\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:23.992697 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:23.992697 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/037031dc-e8e2-48e9-842b-e1e85e0606f4-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:25.621809 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:25.621768 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="037031dc-e8e2-48e9-842b-e1e85e0606f4" path="/var/lib/kubelet/pods/037031dc-e8e2-48e9-842b-e1e85e0606f4/volumes" Nov 04 02:22:30.765798 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:30.765762 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" podUID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" containerName="main" containerID="cri-o://e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73" gracePeriod=45 Nov 04 02:22:31.012895 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.012873 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:22:31.065073 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065040 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065199 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065078 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065199 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065106 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065199 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065165 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\" (UniqueName: \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065317 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065222 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\" (UniqueName: \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065317 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065251 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\") pod \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\" (UID: \"5c756c97-11b8-48f7-9a31-32e7a5f51ae3\") " Nov 04 02:22:31.065400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065331 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:31.065400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065371 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:31.065594 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065571 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:31.065647 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065635 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.065679 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065646 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.065679 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.065659 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.067438 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.067417 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:22:31.067552 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.067455 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:22:31.067552 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.067535 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b" pod "5c756c97-11b8-48f7-9a31-32e7a5f51ae3" (UID: "5c756c97-11b8-48f7-9a31-32e7a5f51ae3"). InnerVolumeSpecName "kube-api-access-wqj2b". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:22:31.166423 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.166343 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.166423 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.166382 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wqj2b\" (UniqueName: \"kubernetes.io/projected/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-kube-api-access-wqj2b\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.166423 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.166397 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/5c756c97-11b8-48f7-9a31-32e7a5f51ae3-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:22:31.747828 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.747792 5029 generic.go:356] "Generic (PLEG): container finished" podID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" containerID="e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73" exitCode=0 Nov 04 02:22:31.747991 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.747856 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" Nov 04 02:22:31.747991 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.747872 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" event={"ID":"5c756c97-11b8-48f7-9a31-32e7a5f51ae3","Type":"ContainerDied","Data":"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73"} Nov 04 02:22:31.747991 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.747923 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx" event={"ID":"5c756c97-11b8-48f7-9a31-32e7a5f51ae3","Type":"ContainerDied","Data":"d3e7fb9572f81208e75a5e82a51fc0d1873aeba6c6ab961044ffd50ce64e1f77"} Nov 04 02:22:31.747991 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.747937 5029 scope.go:122] "RemoveContainer" containerID="e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73" Nov 04 02:22:31.756098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.756078 5029 scope.go:122] "RemoveContainer" containerID="e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73" Nov 04 02:22:31.756363 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:31.756339 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73\": container with ID starting with e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73 not found: ID does not exist" containerID="e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73" Nov 04 02:22:31.756495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.756375 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73"} err="failed to get container status \"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73\": rpc error: code = NotFound desc = could not find container \"e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73\": container with ID starting with e74de32ae94049d224cd6b6b932fb724f15baf4f0967d516d3f06190b5bf2c73 not found: ID does not exist" Nov 04 02:22:31.910971 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.910899 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:22:31.960380 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:31.960355 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w4-196e7e/distinct-w4-196e7e-kserve-5cdcc7f57d-q68hx"] Nov 04 02:22:33.620047 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:22:33.620015 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" path="/var/lib/kubelet/pods/5c756c97-11b8-48f7-9a31-32e7a5f51ae3/volumes" Nov 04 02:22:34.501906 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:22:34.501874 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:23:04.501608 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:23:04.501569 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:23:07.487795 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.487764 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:07.488213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.488160 5029 cpu_manager.go:414] "containerMap: removing container" podUID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" containerName="main" Nov 04 02:23:07.488213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.488175 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" containerName="main" Nov 04 02:23:07.488288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.488236 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="5c756c97-11b8-48f7-9a31-32e7a5f51ae3" containerName="main" Nov 04 02:23:07.491049 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.491028 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.543539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.543518 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-196e7e-openshift-default-dockercfg-cdj7c\"" Nov 04 02:23:07.595575 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595546 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:07.595791 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595769 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.595884 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595818 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.595884 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595850 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.595884 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595878 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.596035 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595913 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.596035 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595940 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.596035 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595960 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ff699\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.596035 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.595994 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.596035 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.596008 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697111 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697079 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697111 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697111 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697146 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697189 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697234 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697271 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697337 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697321 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697373 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697399 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ff699\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697559 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697622 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697639 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.697854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.697730 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.698030 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.698011 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.699654 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.699630 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.699759 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.699682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.758188 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.758131 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.758353 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.758332 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ff699\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\") pod \"e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:07.855268 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.855249 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w4-196e7e-openshift-default-dockercfg-cdj7c\"" Nov 04 02:23:07.864877 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:07.864859 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:08.289631 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.289605 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:08.292100 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:23:08.292070 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e9ffdc4_d417_47d4_9572_e4b9c5024c3f.slice/crio-7506de0e9fec1dac66516bb38930c892492e104fa4748f9c02883e73bdf956b9 WatchSource:0}: Error finding container 7506de0e9fec1dac66516bb38930c892492e104fa4748f9c02883e73bdf956b9: Status 404 returned error can't find the container with id 7506de0e9fec1dac66516bb38930c892492e104fa4748f9c02883e73bdf956b9 Nov 04 02:23:08.294639 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.294612 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:23:08.294717 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.294668 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:23:08.294717 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.294699 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:23:08.915124 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.915089 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" event={"ID":"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f","Type":"ContainerStarted","Data":"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f"} Nov 04 02:23:08.915543 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:08.915137 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" event={"ID":"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f","Type":"ContainerStarted","Data":"7506de0e9fec1dac66516bb38930c892492e104fa4748f9c02883e73bdf956b9"} Nov 04 02:23:09.037588 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:09.037464 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" podStartSLOduration=2.037450253 podStartE2EDuration="2.037450253s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:23:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:23:09.035521681 +0000 UTC m=+1621.528846843" watchObservedRunningTime="2026-11-04 02:23:09.037450253 +0000 UTC m=+1621.530775393" Nov 04 02:23:09.865288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:09.865256 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:09.870597 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:09.870574 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:09.921021 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:09.920995 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:09.921939 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:09.921920 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:17.166449 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:17.166367 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:17.166862 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:17.166605 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" podUID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" containerName="istio-proxy" containerID="cri-o://ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f" gracePeriod=30 Nov 04 02:23:22.413006 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.412974 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:22.532871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.532845 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.532889 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.532906 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.532953 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.532978 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533026 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533022 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533097 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533125 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:22.533303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533178 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533303 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533222 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\") pod \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\" (UID: \"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f\") " Nov 04 02:23:22.533555 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533342 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:22.533555 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533416 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:23:22.533664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533597 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.533664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533619 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.533664 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533634 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.533821 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.533724 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:22.534302 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.534219 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:22.535433 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.535408 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:23:22.535433 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.535418 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:23:22.535759 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.535732 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "kube-api-access-ff699". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:22.535915 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.535893 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token" pod "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" (UID: "3e9ffdc4-d417-47d4-9572-e4b9c5024c3f"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:23:22.634398 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634368 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.634539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634401 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ff699\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-kube-api-access-ff699\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.634539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634417 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.634539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634432 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.634539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634446 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.634539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.634459 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:23:22.979605 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.979510 5029 generic.go:356] "Generic (PLEG): container finished" podID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" containerID="ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f" exitCode=0 Nov 04 02:23:22.979605 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.979561 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" Nov 04 02:23:22.979795 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.979602 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" event={"ID":"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f","Type":"ContainerDied","Data":"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f"} Nov 04 02:23:22.979795 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.979657 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p" event={"ID":"3e9ffdc4-d417-47d4-9572-e4b9c5024c3f","Type":"ContainerDied","Data":"7506de0e9fec1dac66516bb38930c892492e104fa4748f9c02883e73bdf956b9"} Nov 04 02:23:22.979795 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.979678 5029 scope.go:122] "RemoveContainer" containerID="ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f" Nov 04 02:23:22.988498 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.988462 5029 scope.go:122] "RemoveContainer" containerID="ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f" Nov 04 02:23:22.988741 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:23:22.988726 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f\": container with ID starting with ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f not found: ID does not exist" containerID="ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f" Nov 04 02:23:22.988790 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:22.988749 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f"} err="failed to get container status \"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f\": rpc error: code = NotFound desc = could not find container \"ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f\": container with ID starting with ba87486786806b13dfbf6d3011f5ac1a4d6e9590630bed5275c05828d806548f not found: ID does not exist" Nov 04 02:23:23.144157 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:23.144130 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:23.193799 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:23.193779 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w4-196e7e-openshift-default-5b9d665fbc-qf66p"] Nov 04 02:23:23.619169 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:23:23.619127 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" path="/var/lib/kubelet/pods/3e9ffdc4-d417-47d4-9572-e4b9c5024c3f/volumes" Nov 04 02:23:34.501834 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:23:34.501795 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:24:04.502500 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:24:04.502450 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:24:34.501103 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:24:34.501052 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:24:53.183770 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:24:53.183680 5029 prober.go:122] "Probe failed" probeType="Liveness" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal" podUID="3c019a8a489976b5a638075ae287a9c6" containerName="haproxy" probeResult="failure" output="Get \"https://172.20.0.1:6443/version\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 02:25:04.501266 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:25:04.501229 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:25:34.501418 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:25:34.501386 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:26:04.501553 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:26:04.501514 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:26:34.501444 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:26:34.501406 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:27:04.501340 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:27:04.501299 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:27:34.501695 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:27:34.501655 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:28:04.501350 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:04.501256 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:28:27.687551 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:27.687513 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:28:27.737234 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:27.737197 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:28:27.738386 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:27.738363 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:28:27.741777 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:27.741753 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:28:34.502771 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:34.502655 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:28:42.706931 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:42.706863 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" podUID="78ebac77-af85-4c16-a267-0dbc95331e03" containerName="main" containerID="cri-o://100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5" gracePeriod=45 Nov 04 02:28:42.756657 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:42.756620 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" podUID="243ca222-f30f-4837-8651-b489a840dd5b" containerName="main" containerID="cri-o://b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc" gracePeriod=45 Nov 04 02:28:42.757056 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:42.757007 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" podUID="623f84f2-fa46-494a-9ae0-6d20d5660346" containerName="main" containerID="cri-o://746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63" gracePeriod=45 Nov 04 02:28:42.759973 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:42.759944 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" podUID="9113fc57-3a0b-449c-831e-207de47de756" containerName="main" containerID="cri-o://da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b" gracePeriod=45 Nov 04 02:28:43.013686 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.013659 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:28:43.035400 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.035374 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:28:43.050307 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.050274 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:28:43.066307 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.066288 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:28:43.125451 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125419 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125458 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125498 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125523 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125552 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125579 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\" (UniqueName: \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.125621 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125609 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\" (UniqueName: \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125636 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125751 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125775 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125787 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125843 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125867 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125877 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.125932 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125899 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\" (UniqueName: \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125946 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\" (UniqueName: \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.125979 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\" (UniqueName: \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126016 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126050 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\" (UniqueName: \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126077 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126085 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126109 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126326 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126314 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126352 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\" (UniqueName: \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126380 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\" (UniqueName: \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\") pod \"78ebac77-af85-4c16-a267-0dbc95331e03\" (UID: \"78ebac77-af85-4c16-a267-0dbc95331e03\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126425 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\") pod \"243ca222-f30f-4837-8651-b489a840dd5b\" (UID: \"243ca222-f30f-4837-8651-b489a840dd5b\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126465 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126510 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126542 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\") pod \"623f84f2-fa46-494a-9ae0-6d20d5660346\" (UID: \"623f84f2-fa46-494a-9ae0-6d20d5660346\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126585 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\") pod \"9113fc57-3a0b-449c-831e-207de47de756\" (UID: \"9113fc57-3a0b-449c-831e-207de47de756\") " Nov 04 02:28:43.126900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126586 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126960 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126982 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.126998 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127011 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127025 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127038 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127293 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127051 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.127665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127460 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.127665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127512 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.127665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127533 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.127665 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127547 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.127883 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.127817 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.128964 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.128929 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:43.129492 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.129442 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:43.130231 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.130099 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.132086 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.131094 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt" pod "9113fc57-3a0b-449c-831e-207de47de756" (UID: "9113fc57-3a0b-449c-831e-207de47de756"). InnerVolumeSpecName "kube-api-access-t4mpt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:28:43.132086 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.131416 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.132086 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.131515 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n" pod "243ca222-f30f-4837-8651-b489a840dd5b" (UID: "243ca222-f30f-4837-8651-b489a840dd5b"). InnerVolumeSpecName "kube-api-access-fqh6n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:28:43.132086 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.131904 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:43.132544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.132191 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.132544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.132269 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:28:43.132544 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.132324 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "kube-api-access-rcq8t". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:28:43.132702 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.132621 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c" pod "623f84f2-fa46-494a-9ae0-6d20d5660346" (UID: "623f84f2-fa46-494a-9ae0-6d20d5660346"). InnerVolumeSpecName "kube-api-access-vff9c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:28:43.133653 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.133620 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs" pod "78ebac77-af85-4c16-a267-0dbc95331e03" (UID: "78ebac77-af85-4c16-a267-0dbc95331e03"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227827 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fqh6n\" (UniqueName: \"kubernetes.io/projected/243ca222-f30f-4837-8651-b489a840dd5b-kube-api-access-fqh6n\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227858 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vff9c\" (UniqueName: \"kubernetes.io/projected/623f84f2-fa46-494a-9ae0-6d20d5660346-kube-api-access-vff9c\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227869 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227879 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/243ca222-f30f-4837-8651-b489a840dd5b-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227887 5029 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-home\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227896 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/78ebac77-af85-4c16-a267-0dbc95331e03-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.227900 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227904 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/78ebac77-af85-4c16-a267-0dbc95331e03-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227913 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcq8t\" (UniqueName: \"kubernetes.io/projected/78ebac77-af85-4c16-a267-0dbc95331e03-kube-api-access-rcq8t\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227924 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227937 5029 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-model-cache\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227946 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227954 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/623f84f2-fa46-494a-9ae0-6d20d5660346-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227962 5029 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9113fc57-3a0b-449c-831e-207de47de756-tmp-dir\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227969 5029 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/243ca222-f30f-4837-8651-b489a840dd5b-dshm\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227976 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/623f84f2-fa46-494a-9ae0-6d20d5660346-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227983 5029 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9113fc57-3a0b-449c-831e-207de47de756-tls-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.228202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.227991 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t4mpt\" (UniqueName: \"kubernetes.io/projected/9113fc57-3a0b-449c-831e-207de47de756-kube-api-access-t4mpt\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:43.376292 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.376257 5029 generic.go:356] "Generic (PLEG): container finished" podID="623f84f2-fa46-494a-9ae0-6d20d5660346" containerID="746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63" exitCode=0 Nov 04 02:28:43.376437 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.376357 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" Nov 04 02:28:43.376437 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.376363 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" event={"ID":"623f84f2-fa46-494a-9ae0-6d20d5660346","Type":"ContainerDied","Data":"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63"} Nov 04 02:28:43.376437 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.376408 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q" event={"ID":"623f84f2-fa46-494a-9ae0-6d20d5660346","Type":"ContainerDied","Data":"a93147640ea27da5e290e51b4b2f627bd804a6709ffff40ec42ab45942b3eb65"} Nov 04 02:28:43.376437 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.376431 5029 scope.go:122] "RemoveContainer" containerID="746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63" Nov 04 02:28:43.378127 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.378001 5029 generic.go:356] "Generic (PLEG): container finished" podID="78ebac77-af85-4c16-a267-0dbc95331e03" containerID="100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5" exitCode=0 Nov 04 02:28:43.378296 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.378281 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" Nov 04 02:28:43.378599 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.378550 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" event={"ID":"78ebac77-af85-4c16-a267-0dbc95331e03","Type":"ContainerDied","Data":"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5"} Nov 04 02:28:43.378744 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.378720 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9" event={"ID":"78ebac77-af85-4c16-a267-0dbc95331e03","Type":"ContainerDied","Data":"5a0b322770307ab0a8d07d456452ff35ca9670a8d0748bcbcb080e6ce4a9a218"} Nov 04 02:28:43.379980 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.379956 5029 generic.go:356] "Generic (PLEG): container finished" podID="9113fc57-3a0b-449c-831e-207de47de756" containerID="da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b" exitCode=0 Nov 04 02:28:43.380094 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.380064 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" Nov 04 02:28:43.381961 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.381535 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" event={"ID":"9113fc57-3a0b-449c-831e-207de47de756","Type":"ContainerDied","Data":"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b"} Nov 04 02:28:43.381961 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.381573 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" event={"ID":"9113fc57-3a0b-449c-831e-207de47de756","Type":"ContainerDied","Data":"2a89d56f6f413d69d5c399b6bbcb609e7a2e4eba97bd383685dfaa6805b5a4fd"} Nov 04 02:28:43.384501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.383203 5029 generic.go:356] "Generic (PLEG): container finished" podID="243ca222-f30f-4837-8651-b489a840dd5b" containerID="b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc" exitCode=0 Nov 04 02:28:43.384501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.383251 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" Nov 04 02:28:43.384501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.383287 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" event={"ID":"243ca222-f30f-4837-8651-b489a840dd5b","Type":"ContainerDied","Data":"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc"} Nov 04 02:28:43.384501 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.383325 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq" event={"ID":"243ca222-f30f-4837-8651-b489a840dd5b","Type":"ContainerDied","Data":"293c2948d4b403c9fa9b03c2b1e7050e6983fcfd65de7d511973d6ed2488011e"} Nov 04 02:28:43.390915 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.390682 5029 scope.go:122] "RemoveContainer" containerID="746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63" Nov 04 02:28:43.391249 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:43.390998 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63\": container with ID starting with 746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63 not found: ID does not exist" containerID="746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63" Nov 04 02:28:43.391249 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.391031 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63"} err="failed to get container status \"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63\": rpc error: code = NotFound desc = could not find container \"746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63\": container with ID starting with 746397674f39c75c7d10608ef6224f0e0dee86fa56fa794dfbd67a3a16f55d63 not found: ID does not exist" Nov 04 02:28:43.391249 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.391054 5029 scope.go:122] "RemoveContainer" containerID="100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5" Nov 04 02:28:43.401202 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.401180 5029 scope.go:122] "RemoveContainer" containerID="100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5" Nov 04 02:28:43.401493 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:43.401456 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5\": container with ID starting with 100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5 not found: ID does not exist" containerID="100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5" Nov 04 02:28:43.401575 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.401501 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5"} err="failed to get container status \"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5\": rpc error: code = NotFound desc = could not find container \"100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5\": container with ID starting with 100bfa52b6db7e98e6b1487c14921b6bd552943f77f70d0f8e454b96cb9256b5 not found: ID does not exist" Nov 04 02:28:43.401575 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.401522 5029 scope.go:122] "RemoveContainer" containerID="da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b" Nov 04 02:28:43.410548 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.410530 5029 scope.go:122] "RemoveContainer" containerID="da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b" Nov 04 02:28:43.410793 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:43.410776 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b\": container with ID starting with da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b not found: ID does not exist" containerID="da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b" Nov 04 02:28:43.410871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.410801 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b"} err="failed to get container status \"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b\": rpc error: code = NotFound desc = could not find container \"da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b\": container with ID starting with da5d297babf73a66c924f5c15f10c20fa8cd944b77fa7d5a411b9ef7f6b6415b not found: ID does not exist" Nov 04 02:28:43.410871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.410820 5029 scope.go:122] "RemoveContainer" containerID="b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc" Nov 04 02:28:43.418441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.418426 5029 scope.go:122] "RemoveContainer" containerID="b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc" Nov 04 02:28:43.418728 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:43.418710 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc\": container with ID starting with b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc not found: ID does not exist" containerID="b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc" Nov 04 02:28:43.418814 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.418731 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc"} err="failed to get container status \"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc\": rpc error: code = NotFound desc = could not find container \"b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc\": container with ID starting with b6abbb58620d8d27bcd8ffb94f9fbbff5788c662ad40f5389ce09f4762935edc not found: ID does not exist" Nov 04 02:28:43.591807 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.591777 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:28:43.594689 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.594665 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-w2-29e7e8-kserve-6f776ffcbc-rw4rq"] Nov 04 02:28:43.619368 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.619342 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="243ca222-f30f-4837-8651-b489a840dd5b" path="/var/lib/kubelet/pods/243ca222-f30f-4837-8651-b489a840dd5b/volumes" Nov 04 02:28:43.703809 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.703786 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:28:43.714214 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.714187 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/distinct-2-w2-29e7e8-kserve-945878948-92pg9"] Nov 04 02:28:43.822711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.822687 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:28:43.826738 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.826711 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv"] Nov 04 02:28:43.935923 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.935870 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:28:43.939371 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:43.939352 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["e2e-models-e2e-worker-w2-29e7e8/premium-w2-29e7e8-kserve-5c989d5796-5d45q"] Nov 04 02:28:44.014375 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:44.014343 5029 prober.go:122] "Probe failed" probeType="Readiness" pod="e2e-models-e2e-worker-w2-29e7e8/simulator-w2-29e7e8-kserve-66cf9b95b9-f4xmv" podUID="9113fc57-3a0b-449c-831e-207de47de756" containerName="main" probeResult="failure" output="Get \"https://10.133.0.32:8000/ready\": net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)" Nov 04 02:28:45.619087 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:45.619057 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="623f84f2-fa46-494a-9ae0-6d20d5660346" path="/var/lib/kubelet/pods/623f84f2-fa46-494a-9ae0-6d20d5660346/volumes" Nov 04 02:28:45.619485 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:45.619419 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="78ebac77-af85-4c16-a267-0dbc95331e03" path="/var/lib/kubelet/pods/78ebac77-af85-4c16-a267-0dbc95331e03/volumes" Nov 04 02:28:45.619773 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:45.619759 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="9113fc57-3a0b-449c-831e-207de47de756" path="/var/lib/kubelet/pods/9113fc57-3a0b-449c-831e-207de47de756/volumes" Nov 04 02:28:51.111367 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.111333 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:28:51.111831 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.111556 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" podUID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" containerName="maas-api" containerID="cri-o://14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762" gracePeriod=30 Nov 04 02:28:51.371527 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.371446 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:28:51.419156 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.419130 5029 generic.go:356] "Generic (PLEG): container finished" podID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" containerID="14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762" exitCode=0 Nov 04 02:28:51.419267 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.419174 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" Nov 04 02:28:51.419267 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.419222 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" event={"ID":"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0","Type":"ContainerDied","Data":"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762"} Nov 04 02:28:51.419341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.419272 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv" event={"ID":"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0","Type":"ContainerDied","Data":"11e128c56ca4c72f6bbc626bdd77d8b98ee1b586dc14cdf31700fd462354ed4a"} Nov 04 02:28:51.419341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.419283 5029 scope.go:122] "RemoveContainer" containerID="14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762" Nov 04 02:28:51.427291 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.427274 5029 scope.go:122] "RemoveContainer" containerID="14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762" Nov 04 02:28:51.427570 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:28:51.427550 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762\": container with ID starting with 14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762 not found: ID does not exist" containerID="14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762" Nov 04 02:28:51.427619 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.427578 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762"} err="failed to get container status \"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762\": rpc error: code = NotFound desc = could not find container \"14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762\": container with ID starting with 14a06b5a0222609ce7c2a2ba1f80c70ef97cba0feb43a175975d2e2b86df2762 not found: ID does not exist" Nov 04 02:28:51.501562 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.501530 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") pod \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " Nov 04 02:28:51.501676 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.501611 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\") pod \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " Nov 04 02:28:51.501750 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.501693 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\") pod \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " Nov 04 02:28:51.501815 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.501783 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\" (UniqueName: \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\") pod \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\" (UID: \"4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0\") " Nov 04 02:28:51.502435 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.502405 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca" pod "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" (UID: "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:28:51.504868 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.504843 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls" pod "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" (UID: "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:51.505069 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.505047 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5" pod "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" (UID: "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0"). InnerVolumeSpecName "kube-api-access-zptd5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:28:51.508089 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.508054 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs" pod "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" (UID: "4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0"). InnerVolumeSpecName "metrics-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Nov 04 02:28:51.603276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.603250 5029 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-maas-api-tls\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:51.603276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.603274 5029 reconciler_common.go:299] "Volume detached for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-metrics-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:51.603422 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.603284 5029 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-openshift-service-ca\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:51.603422 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.603293 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zptd5\" (UniqueName: \"kubernetes.io/projected/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0-kube-api-access-zptd5\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:28:51.831947 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.831916 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:28:51.835688 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:51.835660 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-worker-w2-29e7e8-595bcf964-p9wpv"] Nov 04 02:28:53.618463 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:28:53.618428 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" path="/var/lib/kubelet/pods/4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0/volumes" Nov 04 02:29:04.501749 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:29:04.501710 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:29:07.369539 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369502 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369843 5029 cpu_manager.go:414] "containerMap: removing container" podUID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" containerName="maas-api" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369855 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" containerName="maas-api" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369864 5029 cpu_manager.go:414] "containerMap: removing container" podUID="78ebac77-af85-4c16-a267-0dbc95331e03" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369869 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="78ebac77-af85-4c16-a267-0dbc95331e03" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369892 5029 cpu_manager.go:414] "containerMap: removing container" podUID="9113fc57-3a0b-449c-831e-207de47de756" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369898 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="9113fc57-3a0b-449c-831e-207de47de756" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369904 5029 cpu_manager.go:414] "containerMap: removing container" podUID="623f84f2-fa46-494a-9ae0-6d20d5660346" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369908 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="623f84f2-fa46-494a-9ae0-6d20d5660346" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369914 5029 cpu_manager.go:414] "containerMap: removing container" podUID="243ca222-f30f-4837-8651-b489a840dd5b" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369918 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="243ca222-f30f-4837-8651-b489a840dd5b" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369930 5029 cpu_manager.go:414] "containerMap: removing container" podUID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" containerName="istio-proxy" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369935 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" containerName="istio-proxy" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369992 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="4667b36a-071c-4d7b-bc3c-b8a31ae2ebb0" containerName="maas-api" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.369999 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="78ebac77-af85-4c16-a267-0dbc95331e03" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.370007 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="9113fc57-3a0b-449c-831e-207de47de756" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.370012 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="623f84f2-fa46-494a-9ae0-6d20d5660346" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.370017 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="243ca222-f30f-4837-8651-b489a840dd5b" containerName="main" Nov 04 02:29:07.370064 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.370023 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="3e9ffdc4-d417-47d4-9572-e4b9c5024c3f" containerName="istio-proxy" Nov 04 02:29:07.372970 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.372953 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.425374 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.425348 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\"" Nov 04 02:29:07.439256 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439228 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439351 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439269 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4w48\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439351 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439291 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439392 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439413 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439429 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439584 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439449 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439584 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439498 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.439584 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.439581 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.476929 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.476903 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:07.540612 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540574 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540621 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w4w48\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540643 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540662 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540687 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540705 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540728 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.540785 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540765 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541043 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.540795 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541088 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.541033 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541178 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.541153 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541346 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.541323 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541441 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.541265 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.541540 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.541454 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.543160 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.543113 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.543609 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.543591 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.601605 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.601581 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.601871 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.601851 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4w48\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\") pod \"e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:07.736605 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.736576 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"e2e-worker-w2-29e7e8-openshift-default-dockercfg-7ntvz\"" Nov 04 02:29:07.746452 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:07.746430 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:08.197084 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.197063 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:08.200004 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:29:08.199978 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64a5ff3b_9464_4f06_8bc6_b0fd42a2ed15.slice/crio-9c69e623ba6d2c0ec984ba870a4277b63ca5b9b6b8910b360a6bfaba5e00aafe WatchSource:0}: Error finding container 9c69e623ba6d2c0ec984ba870a4277b63ca5b9b6b8910b360a6bfaba5e00aafe: Status 404 returned error can't find the container with id 9c69e623ba6d2c0ec984ba870a4277b63ca5b9b6b8910b360a6bfaba5e00aafe Nov 04 02:29:08.202132 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.202084 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:29:08.202247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.202162 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:29:08.202247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.202209 5029 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892012Ki","pods":"250"} Nov 04 02:29:08.496791 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.496760 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" event={"ID":"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15","Type":"ContainerStarted","Data":"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7"} Nov 04 02:29:08.497279 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.496816 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" event={"ID":"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15","Type":"ContainerStarted","Data":"9c69e623ba6d2c0ec984ba870a4277b63ca5b9b6b8910b360a6bfaba5e00aafe"} Nov 04 02:29:08.618213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.618160 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podStartSLOduration=1.6181476959999999 podStartE2EDuration="1.618147696s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:29:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:29:08.617113848 +0000 UTC m=+1981.110439006" watchObservedRunningTime="2026-11-04 02:29:08.618147696 +0000 UTC m=+1981.111472885" Nov 04 02:29:08.762556 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:08.761571 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:09.763495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:09.762560 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:29:09.763495 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:09.762619 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:29:10.758497 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:10.755895 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:29:10.758497 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:10.755961 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:29:11.762494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:11.759835 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:29:11.762494 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:11.759896 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:29:12.758496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:12.757567 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:29:12.758496 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:12.757623 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:29:13.758236 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:13.757811 5029 patch_prober.go:29] interesting pod/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4 container/istio-proxy namespace/openshift-ingress: Startup probe status=failure output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" start-of-body= Nov 04 02:29:13.758236 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:13.757870 5029 prober.go:122] "Probe failed" probeType="Startup" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" probeResult="failure" output="Get \"http://10.133.0.51:15021/healthz/ready\": dial tcp 10.133.0.51:15021: connect: connection refused" Nov 04 02:29:14.751109 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:14.751078 5029 kubelet.go:2727] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:14.751445 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:14.751425 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:14.752180 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:14.752161 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:18.528424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:18.528337 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:18.528910 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:18.528593 5029 kuberuntime_container.go:898] "Killing container with a grace period" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" containerID="cri-o://2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7" gracePeriod=30 Nov 04 02:29:20.306612 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:20.306586 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nfhp7_0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d/manager/0.log" Nov 04 02:29:20.534543 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:20.534520 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hf7tr_a3053351-606b-4a29-ae0f-34205a984afb/kuadrant-console-plugin/0.log" Nov 04 02:29:23.769912 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.769886 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:23.879236 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.879159 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gkgm9_8b56c643-0cf2-4293-b1a1-fb9fde73954c/manager/0.log" Nov 04 02:29:23.899721 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899694 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.899854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899745 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.899854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899788 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.899854 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899841 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899873 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899909 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899978 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900020 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.899999 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "istio-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:29:23.900213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900043 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900102 5029 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\") pod \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\" (UID: \"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15\") " Nov 04 02:29:23.900213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900109 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "workload-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:29:23.900213 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900178 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "workload-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:29:23.900424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900275 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "credential-socket". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:29:23.900424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900277 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "istiod-ca-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Nov 04 02:29:23.900424 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900411 5029 reconciler_common.go:299] "Volume detached for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istiod-ca-cert\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:23.900600 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900427 5029 reconciler_common.go:299] "Volume detached for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-credential-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:23.900600 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900436 5029 reconciler_common.go:299] "Volume detached for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-data\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:23.900600 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900444 5029 reconciler_common.go:299] "Volume detached for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-socket\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:23.900600 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.900456 5029 reconciler_common.go:299] "Volume detached for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-workload-certs\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:23.902052 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.902022 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "istio-podinfo". PluginName "kubernetes.io/downward-api", VolumeGIDValue "" Nov 04 02:29:23.902221 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.902199 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "kube-api-access-w4w48". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:29:23.902601 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.902582 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "istio-envoy". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Nov 04 02:29:23.902670 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:23.902627 5029 operation_generator.go:780] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token" pod "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" (UID: "64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Nov 04 02:29:24.001039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.000987 5029 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4w48\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-kube-api-access-w4w48\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:24.001039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.001034 5029 reconciler_common.go:299] "Volume detached for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-envoy\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:24.001039 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.001047 5029 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-token\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:24.001250 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.001061 5029 reconciler_common.go:299] "Volume detached for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15-istio-podinfo\") on node \"ip-10-0-134-176.us-west-1.compute.internal\" DevicePath \"\"" Nov 04 02:29:24.562331 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.562301 5029 generic.go:356] "Generic (PLEG): container finished" podID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerID="2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7" exitCode=0 Nov 04 02:29:24.562513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.562352 5029 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" Nov 04 02:29:24.562513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.562381 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" event={"ID":"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15","Type":"ContainerDied","Data":"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7"} Nov 04 02:29:24.562513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.562432 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4" event={"ID":"64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15","Type":"ContainerDied","Data":"9c69e623ba6d2c0ec984ba870a4277b63ca5b9b6b8910b360a6bfaba5e00aafe"} Nov 04 02:29:24.562513 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.562455 5029 scope.go:122] "RemoveContainer" containerID="2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7" Nov 04 02:29:24.572928 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.572910 5029 scope.go:122] "RemoveContainer" containerID="2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7" Nov 04 02:29:24.573277 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:29:24.573259 5029 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7\": container with ID starting with 2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7 not found: ID does not exist" containerID="2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7" Nov 04 02:29:24.573373 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.573288 5029 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7"} err="failed to get container status \"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7\": rpc error: code = NotFound desc = could not find container \"2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7\": container with ID starting with 2ec3714bea8db1edb043bdb3802a8d880de7f77139f9a8fee748220e4a2a48b7 not found: ID does not exist" Nov 04 02:29:24.726965 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.726941 5029 kubelet.go:2624] "SyncLoop DELETE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:24.776754 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:24.776731 5029 kubelet.go:2618] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/e2e-worker-w2-29e7e8-openshift-default-5d7f77986f-2mxb4"] Nov 04 02:29:25.618434 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:29:25.618404 5029 kubelet_volumes.go:161] "Cleaned up orphaned pod volumes dir" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" path="/var/lib/kubelet/pods/64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15/volumes" Nov 04 02:29:34.501632 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:29:34.501595 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:30:02.660276 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:02.660246 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gkgm9_8b56c643-0cf2-4293-b1a1-fb9fde73954c/manager/0.log" Nov 04 02:30:04.501735 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:30:04.501694 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:30:15.064599 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:15.064563 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-5cf77b77f7-zsw8g_90875d56-cf9b-402f-9bd5-fcb993cf91d3/maas-api/0.log" Nov 04 02:30:19.420831 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:19.420804 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nfhp7_0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d/manager/0.log" Nov 04 02:30:19.640051 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:19.640027 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hf7tr_a3053351-606b-4a29-ae0f-34205a984afb/kuadrant-console-plugin/0.log" Nov 04 02:30:22.351711 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:22.351681 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77fbbd9bf8-5zczj_8f4bbf7f-ffc0-4baf-adce-2c77363ba644/kube-auth-proxy/0.log" Nov 04 02:30:33.100678 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:33.100646 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-8rt47_3d139de1-dfac-49f1-a95d-4f2470c81a89/istio-proxy/0.log" Nov 04 02:30:33.804710 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:33.804675 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj_f42723eb-4c4b-490d-82d0-4fa690ee2a8a/istio-proxy/0.log" Nov 04 02:30:34.501227 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:30:34.501191 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:30:34.760914 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:34.760836 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf_fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62/istio-proxy/0.log" Nov 04 02:30:49.556969 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.556892 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-9w9jd/must-gather-6k75h"] Nov 04 02:30:49.557456 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.557283 5029 cpu_manager.go:414] "containerMap: removing container" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" Nov 04 02:30:49.557456 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.557296 5029 state_mem.go:113] "Deleted CPUSet assignment" logger="CPUManager state memory" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" Nov 04 02:30:49.557456 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.557369 5029 memory_manager.go:365] "RemoveStaleState removing state" podUID="64a5ff3b-9464-4f06-8bc6-b0fd42a2ed15" containerName="istio-proxy" Nov 04 02:30:49.559936 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.559920 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.608041 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.608017 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8f8a4386-ca0f-4f18-9975-6578286093a0-must-gather-output\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.608162 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.608072 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9ppg\" (UniqueName: \"kubernetes.io/projected/8f8a4386-ca0f-4f18-9975-6578286093a0-kube-api-access-w9ppg\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.620034 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.620010 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w9jd/must-gather-6k75h"] Nov 04 02:30:49.661926 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.661901 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w9jd\"/\"kube-root-ca.crt\"" Nov 04 02:30:49.662057 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.662003 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-9w9jd\"/\"default-dockercfg-4clmb\"" Nov 04 02:30:49.662112 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.662097 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-9w9jd\"/\"openshift-service-ca.crt\"" Nov 04 02:30:49.709132 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.709107 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w9ppg\" (UniqueName: \"kubernetes.io/projected/8f8a4386-ca0f-4f18-9975-6578286093a0-kube-api-access-w9ppg\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.709324 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.709306 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8f8a4386-ca0f-4f18-9975-6578286093a0-must-gather-output\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.709704 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.709682 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/8f8a4386-ca0f-4f18-9975-6578286093a0-must-gather-output\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.770031 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.769998 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9ppg\" (UniqueName: \"kubernetes.io/projected/8f8a4386-ca0f-4f18-9975-6578286093a0-kube-api-access-w9ppg\") pod \"must-gather-6k75h\" (UID: \"8f8a4386-ca0f-4f18-9975-6578286093a0\") " pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:49.873030 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:49.872940 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-9w9jd/must-gather-6k75h" Nov 04 02:30:50.243938 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:50.243913 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-9w9jd/must-gather-6k75h"] Nov 04 02:30:50.246343 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:30:50.246315 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8f8a4386_ca0f_4f18_9975_6578286093a0.slice/crio-4e235c0377a90d97d148344fd19cdd4bbba7c75b0d9078c29aa02252117ba5bc WatchSource:0}: Error finding container 4e235c0377a90d97d148344fd19cdd4bbba7c75b0d9078c29aa02252117ba5bc: Status 404 returned error can't find the container with id 4e235c0377a90d97d148344fd19cdd4bbba7c75b0d9078c29aa02252117ba5bc Nov 04 02:30:50.248343 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:50.248328 5029 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Nov 04 02:30:50.906270 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:50.906232 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w9jd/must-gather-6k75h" event={"ID":"8f8a4386-ca0f-4f18-9975-6578286093a0","Type":"ContainerStarted","Data":"4e235c0377a90d97d148344fd19cdd4bbba7c75b0d9078c29aa02252117ba5bc"} Nov 04 02:30:58.943571 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:58.943538 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w9jd/must-gather-6k75h" event={"ID":"8f8a4386-ca0f-4f18-9975-6578286093a0","Type":"ContainerStarted","Data":"18592df999730744320436708255834cdcb8e1ca46e7f7522684ec68adf3b41f"} Nov 04 02:30:58.943571 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:58.943576 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-9w9jd/must-gather-6k75h" event={"ID":"8f8a4386-ca0f-4f18-9975-6578286093a0","Type":"ContainerStarted","Data":"d87b55d72a273542b198710095961eaa6445169570fc2b5b120c8a73dfd68a03"} Nov 04 02:30:59.062246 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:59.062191 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-9w9jd/must-gather-6k75h" podStartSLOduration=2.2769663749999998 podStartE2EDuration="10.062178596s" totalImagesPullingTime="7.785212221s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-11-04 02:30:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:30:59.061304857 +0000 UTC m=+2091.554629989" watchObservedRunningTime="2026-11-04 02:30:59.062178596 +0000 UTC m=+2091.555503768" Nov 04 02:30:59.094901 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:30:59.094876 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-w2257_38e16a3b-bcac-4482-b177-eb0061845a74/global-pull-secret-syncer/0.log" Nov 04 02:31:02.058867 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:02.058837 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-75hc6_447db221-9de8-4e44-b228-ab76255a229d/konnectivity-agent/0.log" Nov 04 02:31:02.917567 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:02.917540 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-176.us-west-1.compute.internal_3c019a8a489976b5a638075ae287a9c6/haproxy/0.log" Nov 04 02:31:04.501913 ip-10-0-134-176 kubenswrapper[5029]: E1104 02:31:04.501866 5029 prober.go:265] "Unable to write all bytes from execInContainer" err="short write" expectedBytes=11141 actualBytes=10240 pod="openshift-insights/insights-runtime-extractor-mrgm4" containerName="extractor" Nov 04 02:31:08.766098 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.766065 5029 kubelet.go:2608] "SyncLoop ADD" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv"] Nov 04 02:31:08.775072 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.775049 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.827172 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.827140 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"kube-root-ca.crt\"" Nov 04 02:31:08.827327 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.827229 5029 reflector.go:446] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-w2sfr\"/\"openshift-service-ca.crt\"" Nov 04 02:31:08.875620 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.875585 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-podres\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.875620 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.875620 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tklgr\" (UniqueName: \"kubernetes.io/projected/bbe6f585-3730-474f-a0c9-3741d0fccbcd-kube-api-access-tklgr\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.875870 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.875638 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-lib-modules\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.875870 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.875743 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-proc\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.875870 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.875861 5029 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-sys\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.876991 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.876968 5029 reflector.go:446] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-w2sfr\"/\"default-dockercfg-lvbsf\"" Nov 04 02:31:08.878288 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.878267 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv"] Nov 04 02:31:08.976583 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976550 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-sys\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976763 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976610 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-podres\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976763 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976687 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-podres\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976763 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976704 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-sys\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976763 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976726 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tklgr\" (UniqueName: \"kubernetes.io/projected/bbe6f585-3730-474f-a0c9-3741d0fccbcd-kube-api-access-tklgr\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976775 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-lib-modules\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976810 5029 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-proc\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976833 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-lib-modules\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:08.976920 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:08.976882 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/bbe6f585-3730-474f-a0c9-3741d0fccbcd-proc\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:09.038244 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.038170 5029 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tklgr\" (UniqueName: \"kubernetes.io/projected/bbe6f585-3730-474f-a0c9-3741d0fccbcd-kube-api-access-tklgr\") pod \"perf-node-gather-daemonset-lwpqv\" (UID: \"bbe6f585-3730-474f-a0c9-3741d0fccbcd\") " pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:09.086109 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.086075 5029 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:09.436144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.436055 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-8557cbdb84-nfhp7_0ccae54d-06a6-4d0c-8dd1-55fdfcf7cb1d/manager/0.log" Nov 04 02:31:09.459855 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.459833 5029 kubelet.go:2615] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv"] Nov 04 02:31:09.462403 ip-10-0-134-176 kubenswrapper[5029]: W1104 02:31:09.462370 5029 manager.go:1172] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podbbe6f585_3730_474f_a0c9_3741d0fccbcd.slice/crio-b5e1147ade1a1eed7d334bcccd89c8cad6ccb7ff7048c3d81c8b9339b56ffa24 WatchSource:0}: Error finding container b5e1147ade1a1eed7d334bcccd89c8cad6ccb7ff7048c3d81c8b9339b56ffa24: Status 404 returned error can't find the container with id b5e1147ade1a1eed7d334bcccd89c8cad6ccb7ff7048c3d81c8b9339b56ffa24 Nov 04 02:31:09.857111 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.857082 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66c4fb575d-hf7tr_a3053351-606b-4a29-ae0f-34205a984afb/kuadrant-console-plugin/0.log" Nov 04 02:31:09.991543 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.991508 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" event={"ID":"bbe6f585-3730-474f-a0c9-3741d0fccbcd","Type":"ContainerStarted","Data":"182f9f233242b0f6b8f6cbc92d24ce2e78166e264fcac4f44b40eabe1537a4bc"} Nov 04 02:31:09.991737 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.991561 5029 kubelet.go:2639] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" event={"ID":"bbe6f585-3730-474f-a0c9-3741d0fccbcd","Type":"ContainerStarted","Data":"b5e1147ade1a1eed7d334bcccd89c8cad6ccb7ff7048c3d81c8b9339b56ffa24"} Nov 04 02:31:09.991737 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:09.991651 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:10.107789 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:10.107705 5029 pod_startup_latency_tracker.go:148] "Observed pod startup duration" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" podStartSLOduration=2.107693705 podStartE2EDuration="2.107693705s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-11-04 02:31:08 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-11-04 02:31:10.106823856 +0000 UTC m=+2102.600149000" watchObservedRunningTime="2026-11-04 02:31:10.107693705 +0000 UTC m=+2102.601018844" Nov 04 02:31:12.595615 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:12.595582 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gkgm9_8b56c643-0cf2-4293-b1a1-fb9fde73954c/manager/0.log" Nov 04 02:31:16.005792 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:16.005764 5029 kubelet.go:2727] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-w2sfr/perf-node-gather-daemonset-lwpqv" Nov 04 02:31:17.226176 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:17.226142 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gkgm9_8b56c643-0cf2-4293-b1a1-fb9fde73954c/manager/0.log" Nov 04 02:31:19.543247 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:19.543219 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2mst5_ddb170ac-1121-452d-bf81-cef8e8e2e3d1/node-exporter/0.log" Nov 04 02:31:19.961397 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:19.961313 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2mst5_ddb170ac-1121-452d-bf81-cef8e8e2e3d1/kube-rbac-proxy/0.log" Nov 04 02:31:20.381050 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:20.381021 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-2mst5_ddb170ac-1121-452d-bf81-cef8e8e2e3d1/init-textfile/0.log" Nov 04 02:31:21.883144 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:21.883111 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-6466c554d8-gkgm9_8b56c643-0cf2-4293-b1a1-fb9fde73954c/manager/0.log" Nov 04 02:31:24.624231 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:24.624202 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/prometheus/0.log" Nov 04 02:31:25.045817 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:25.045791 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/config-reloader/0.log" Nov 04 02:31:25.471924 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:25.471851 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/thanos-sidecar/0.log" Nov 04 02:31:25.900131 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:25.900100 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/kube-rbac-proxy-web/0.log" Nov 04 02:31:26.321396 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:26.321371 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/kube-rbac-proxy/0.log" Nov 04 02:31:26.740873 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:26.740845 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/kube-rbac-proxy-thanos/0.log" Nov 04 02:31:27.164392 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:27.164318 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_04c8d696-35f9-4ade-ac3b-9bb205e7d34a/init-config-reloader/0.log" Nov 04 02:31:27.529344 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:27.529310 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-77fbbd9bf8-5zczj_8f4bbf7f-ffc0-4baf-adce-2c77363ba644/kube-auth-proxy/0.log" Nov 04 02:31:27.605385 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:27.605359 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-ppdcg_7299f7d4-b1ef-4a5d-a8fd-883550532d2c/prometheus-operator/0.log" Nov 04 02:31:28.028807 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:28.028779 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-9cd89b74c-ppdcg_7299f7d4-b1ef-4a5d-a8fd-883550532d2c/kube-rbac-proxy/0.log" Nov 04 02:31:28.168855 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:28.168830 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-8rt47_3d139de1-dfac-49f1-a95d-4f2470c81a89/istio-proxy/0.log" Nov 04 02:31:28.814354 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:28.814322 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-kdsgj_f42723eb-4c4b-490d-82d0-4fa690ee2a8a/istio-proxy/0.log" Nov 04 02:31:29.570341 ip-10-0-134-176 kubenswrapper[5029]: I1104 02:31:29.570314 5029 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7bdc57c4cf-rfgdf_fdb391b4-37c6-48a7-9fa2-a7f67ed4ee62/istio-proxy/0.log"