Jul 13 20:19:13.730022 ip-10-0-128-52 systemd[1]: Starting Kubernetes Kubelet... Jul 13 20:19:14.085974 ip-10-0-128-52 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:19:14.085974 ip-10-0-128-52 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 13 20:19:14.085974 ip-10-0-128-52 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:19:14.085974 ip-10-0-128-52 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 13 20:19:14.085974 ip-10-0-128-52 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 13 20:19:14.087718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.087636 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 13 20:19:14.092110 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092089 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.092110 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092108 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.092110 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092114 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092119 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092123 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092126 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092129 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092131 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092134 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092136 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092139 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092142 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092145 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092147 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092150 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092153 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092155 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092159 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092161 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092164 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092166 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.092253 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092169 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092171 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092174 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092176 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092179 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092181 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092184 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092189 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092192 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092194 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092196 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092199 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092201 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092204 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092206 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092210 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092213 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092215 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092218 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092220 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.092699 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092222 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092225 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092227 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092230 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092232 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092235 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092237 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092239 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092242 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092244 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092246 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092249 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092251 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092253 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092256 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092258 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092261 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092264 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092267 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092269 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.093227 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092271 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092274 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092276 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092279 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092281 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092283 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092287 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092290 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092293 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092295 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092298 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092301 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092303 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092306 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092308 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092312 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092315 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092318 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092321 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.093702 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092323 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092326 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092329 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092332 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092334 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092337 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092339 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092343 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092346 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092349 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092352 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092354 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092356 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092359 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092361 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092363 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092365 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092368 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.092370 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093070 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.094202 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093077 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093081 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093084 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093087 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093089 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093092 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093095 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093098 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093100 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093103 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093106 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093108 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093111 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093113 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093115 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093118 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093121 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093123 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093126 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093128 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.094706 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093131 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093133 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093136 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093139 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093142 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093146 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093149 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093151 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093155 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093157 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093160 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093162 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093166 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093169 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093171 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093174 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093177 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093179 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093182 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093185 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.095237 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093188 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093190 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093192 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093195 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093198 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093201 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093203 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093205 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093208 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093211 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093213 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093216 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093218 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093220 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093223 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093225 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093229 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093231 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093234 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093236 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.095756 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093239 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093241 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093244 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093246 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093251 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093254 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093256 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093259 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093262 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093265 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093268 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093270 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093273 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093275 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093279 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093281 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093284 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093286 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093288 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093291 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.096261 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093293 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093296 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093299 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093301 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093304 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093307 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093309 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093312 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093314 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093317 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093319 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093321 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093324 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093326 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093328 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093331 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093335 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093339 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093691 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.096765 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093696 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093699 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093701 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093705 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093707 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093710 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093713 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093715 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093718 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093720 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093722 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093725 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093729 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093732 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093735 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093756 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093759 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093762 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093765 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093767 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.097243 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093770 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093774 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093777 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093779 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093782 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093785 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093787 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093791 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093795 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093799 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093802 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093805 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093808 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093811 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093813 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093816 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093818 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093821 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093823 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.097722 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093826 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093828 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093830 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093833 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093835 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093838 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093840 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093843 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093845 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093848 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093851 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093853 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093855 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093858 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093861 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093863 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093866 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093868 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093870 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093873 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.098233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093875 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093877 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093882 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093884 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093887 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093889 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093892 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093894 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093897 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093899 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093901 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093904 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093906 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093909 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093911 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093913 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093916 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093918 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093922 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093925 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.098714 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093927 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093930 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093933 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093935 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093937 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093940 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093944 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093947 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093949 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093953 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093956 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093958 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093961 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093963 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093966 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093969 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093971 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093974 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.093976 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094045 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 13 20:19:14.099212 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094056 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094062 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094065 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094069 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094073 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094077 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094081 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094084 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094087 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094090 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094093 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094096 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094098 2580 flags.go:64] FLAG: --cgroup-root="" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094101 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094104 2580 flags.go:64] FLAG: --client-ca-file="" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094108 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094111 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094122 2580 flags.go:64] FLAG: --cluster-domain="" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094126 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094129 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094132 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094135 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094142 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 13 20:19:14.099699 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094145 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094148 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094151 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094157 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094160 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094163 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094166 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094169 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094173 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094176 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094179 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094181 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094184 2580 flags.go:64] FLAG: --enable-server="true" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094187 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094191 2580 flags.go:64] FLAG: --event-burst="100" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094194 2580 flags.go:64] FLAG: --event-qps="50" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094196 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094199 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094202 2580 flags.go:64] FLAG: --eviction-hard="" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094210 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094213 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094216 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094219 2580 flags.go:64] FLAG: --eviction-soft="" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094222 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094225 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 13 20:19:14.100299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094228 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094231 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094236 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094238 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094241 2580 flags.go:64] FLAG: --feature-gates="" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094244 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094247 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094250 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094253 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094255 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094259 2580 flags.go:64] FLAG: --help="false" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094262 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094265 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094268 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094270 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094273 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094276 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094279 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094281 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094284 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094287 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094289 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094292 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094295 2580 flags.go:64] FLAG: --kube-reserved="" Jul 13 20:19:14.100973 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094298 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094300 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094303 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094306 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094308 2580 flags.go:64] FLAG: --lock-file="" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094311 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094313 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094316 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094321 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094328 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094332 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094335 2580 flags.go:64] FLAG: --logging-format="text" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094337 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094340 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094343 2580 flags.go:64] FLAG: --manifest-url="" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094346 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094349 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094352 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094358 2580 flags.go:64] FLAG: --max-pods="110" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094361 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094364 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094367 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094369 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094372 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094375 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 13 20:19:14.101563 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094378 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094385 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094388 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094391 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094394 2580 flags.go:64] FLAG: --pod-cidr="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094396 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094401 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094404 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094407 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094410 2580 flags.go:64] FLAG: --port="10250" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094413 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094415 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-095e25183d843770a" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094419 2580 flags.go:64] FLAG: --qos-reserved="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094421 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094424 2580 flags.go:64] FLAG: --register-node="true" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094427 2580 flags.go:64] FLAG: --register-with-taints="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094430 2580 flags.go:64] FLAG: --registry-burst="10" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094434 2580 flags.go:64] FLAG: --registry-qps="5" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094437 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094439 2580 flags.go:64] FLAG: --reserved-memory="" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094443 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094446 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094448 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094451 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094454 2580 flags.go:64] FLAG: --runonce="false" Jul 13 20:19:14.102185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094456 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094460 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094463 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094466 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094468 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094471 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094474 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094477 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094480 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094482 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094485 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094488 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094491 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094494 2580 flags.go:64] FLAG: --system-cgroups="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094496 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094501 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094504 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094506 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094511 2580 flags.go:64] FLAG: --tls-min-version="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094513 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094515 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094518 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094521 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094524 2580 flags.go:64] FLAG: --v="2" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094533 2580 flags.go:64] FLAG: --version="false" Jul 13 20:19:14.102886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094537 2580 flags.go:64] FLAG: --vmodule="" Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094541 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094544 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094653 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094657 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094660 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094664 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094668 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094672 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094675 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094678 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094681 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094683 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094686 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094688 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094691 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094694 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094696 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094698 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.103501 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094701 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094703 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094706 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094710 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094713 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094716 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094719 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094722 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094724 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094727 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094730 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094732 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094736 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094752 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094755 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094758 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094761 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094764 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094768 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094771 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.104000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094773 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094777 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094780 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094782 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094785 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094788 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094790 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094793 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094795 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094798 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094800 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094803 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094805 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094807 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094810 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094813 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094815 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094817 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094820 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094822 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.104527 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094824 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094827 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094829 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094831 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094835 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094838 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094841 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094843 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094846 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094848 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094851 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094853 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094855 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094859 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094862 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094864 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094866 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094869 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094871 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.105060 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094874 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094876 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094879 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094881 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094883 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094886 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094888 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094890 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094893 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094895 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094897 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094900 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094902 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094904 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094907 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094909 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094912 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094915 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094918 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094921 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.105525 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094923 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094926 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094928 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.094931 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.094937 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.101891 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.101906 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.102680 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.102687 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102778 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102783 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102787 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102790 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102794 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102797 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.106113 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102800 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102802 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102805 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102807 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102810 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102812 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102816 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102820 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102823 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102825 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102828 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102831 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102834 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102836 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102839 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102841 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102844 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102846 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102850 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.106494 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102852 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102855 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102857 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102860 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102863 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102865 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102867 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102870 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102872 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102875 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102877 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102879 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102882 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102884 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102887 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102890 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102892 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102895 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102898 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.106980 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102902 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102905 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102908 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102911 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102914 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102917 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102919 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102921 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102924 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102926 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102928 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102931 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102933 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102942 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102945 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102948 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102950 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102953 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102955 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102958 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.107701 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102960 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102963 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102966 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102968 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102971 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102973 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102975 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102978 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102980 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102983 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102985 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102988 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102990 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102993 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102995 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.102997 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103000 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103004 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103007 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.108445 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103010 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103013 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103016 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103019 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103022 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103025 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103027 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103036 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103039 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103041 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103044 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103047 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103050 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103052 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103055 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.108972 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103057 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.103062 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103206 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103212 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103215 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103218 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103221 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103223 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103226 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103229 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103231 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103233 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103236 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103238 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 13 20:19:14.109502 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103240 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103243 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103245 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103248 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103250 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103252 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103255 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103258 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103261 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103264 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103267 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103276 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103279 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103281 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103284 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103287 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103289 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103291 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103294 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 13 20:19:14.109877 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103296 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103299 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103301 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103304 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103306 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103308 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103311 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103313 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103315 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103318 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103320 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103322 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103325 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103327 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103329 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103332 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103334 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103336 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103339 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103341 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 13 20:19:14.110498 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103344 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103346 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103348 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103351 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103353 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103361 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103364 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103366 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103369 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103371 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103374 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103376 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103379 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103381 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103384 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103386 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103389 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103391 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103394 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103396 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 13 20:19:14.111123 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103399 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103401 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103405 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103408 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103411 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103414 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103416 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103419 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103422 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103424 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103426 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103429 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103431 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103433 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103436 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103438 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103446 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103449 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103459 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103462 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 13 20:19:14.111619 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103464 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103467 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103469 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103472 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103474 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103476 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103479 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:14.103481 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.103486 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.103596 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.105782 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.107050 2580 server.go:1019] "Starting client certificate rotation" Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.107170 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:19:14.112160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.107223 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 13 20:19:14.112499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.110213 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:19:14.112499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.111989 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 13 20:19:14.122565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.122543 2580 log.go:25] "Validated CRI v1 runtime API" Jul 13 20:19:14.124634 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.124621 2580 log.go:25] "Validated CRI v1 image API" Jul 13 20:19:14.125376 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.125358 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 13 20:19:14.128996 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.128976 2580 fs.go:135] Filesystem UUIDs: map[0909c220-91a7-4db8-a1cf-095bbee5e2c7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 7e055030-000f-4506-9e4e-9100fe69e9d2:/dev/nvme0n1p3] Jul 13 20:19:14.129086 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.128993 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 13 20:19:14.133008 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.132899 2580 manager.go:216] Machine: {Timestamp:2026-07-13 20:19:14.132428642 +0000 UTC m=+0.296987983 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3107287 MemoryCapacity:32812171264 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2e694a9a6712a449fb11bd99e79f5c SystemUUID:ec2e694a-9a67-12a4-49fb-11bd99e79f5c BootID:62c2cced-bff8-41ca-9a90-a6f0065a3521 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:12:c2:22:d8:51 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:12:c2:22:d8:51 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ca:c7:cc:b8:bc:64 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812171264 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 13 20:19:14.133008 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.132998 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 13 20:19:14.133109 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.133062 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 13 20:19:14.134610 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.134597 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:19:14.136016 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.135993 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 13 20:19:14.136148 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.136020 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-52.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 13 20:19:14.136777 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.136766 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 13 20:19:14.136818 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.136780 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 13 20:19:14.136818 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.136787 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:19:14.137431 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.137422 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 13 20:19:14.138904 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.138894 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 13 20:19:14.142855 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.142844 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:19:14.142958 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.142949 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 13 20:19:14.142990 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.142978 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 13 20:19:14.142990 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.142987 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 13 20:19:14.143045 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.143000 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 13 20:19:14.143045 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.143009 2580 kubelet.go:398] "Adding apiserver pod source" Jul 13 20:19:14.143045 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.143017 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 13 20:19:14.143399 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.143388 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:19:14.143447 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.143422 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 13 20:19:14.147246 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.147229 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 13 20:19:14.149195 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149178 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 13 20:19:14.149284 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149218 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 13 20:19:14.149419 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149407 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 13 20:19:14.149484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149426 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 13 20:19:14.149484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149435 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 13 20:19:14.149484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149457 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 13 20:19:14.149484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149471 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 13 20:19:14.149484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149479 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 13 20:19:14.149911 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149900 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 13 20:19:14.149969 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149913 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 13 20:19:14.149969 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149924 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 13 20:19:14.149969 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149934 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 13 20:19:14.149969 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149961 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 13 20:19:14.150133 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.149973 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 13 20:19:14.150133 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.150006 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 13 20:19:14.150133 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.150015 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 13 20:19:14.153114 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.153097 2580 server.go:1268] "Started kubelet" Jul 13 20:19:14.153262 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.153234 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 13 20:19:14.153351 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.153190 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 13 20:19:14.153351 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.153303 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 13 20:19:14.153992 ip-10-0-128-52 systemd[1]: Started Kubernetes Kubelet. Jul 13 20:19:14.156218 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.156138 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 13 20:19:14.156373 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.156324 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 13 20:19:14.156431 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.156423 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-52.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 13 20:19:14.156623 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.156602 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 13 20:19:14.157886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.157868 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 13 20:19:14.160925 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.160895 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 13 20:19:14.161823 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.161808 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wc65" Jul 13 20:19:14.163172 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163078 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 13 20:19:14.163172 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163125 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 13 20:19:14.163378 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.163299 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.163844 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163811 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 13 20:19:14.163844 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163826 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 13 20:19:14.163993 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163918 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 13 20:19:14.163993 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163938 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 13 20:19:14.163993 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.163948 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 13 20:19:14.164176 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.163005 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-52.ec2.internal.18c1f2dc01d7c544 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-52.ec2.internal,UID:ip-10-0-128-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-52.ec2.internal,},FirstTimestamp:2026-07-13 20:19:14.15307194 +0000 UTC m=+0.317631282,LastTimestamp:2026-07-13 20:19:14.15307194 +0000 UTC m=+0.317631282,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-52.ec2.internal,}" Jul 13 20:19:14.164386 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164364 2580 factory.go:55] Registering systemd factory Jul 13 20:19:14.164386 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164389 2580 factory.go:223] Registration of the systemd container factory successfully Jul 13 20:19:14.164619 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164604 2580 factory.go:153] Registering CRI-O factory Jul 13 20:19:14.164619 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164623 2580 factory.go:223] Registration of the crio container factory successfully Jul 13 20:19:14.164755 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164694 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 13 20:19:14.164755 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164719 2580 factory.go:103] Registering Raw factory Jul 13 20:19:14.164755 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.164736 2580 manager.go:1196] Started watching for new ooms in manager Jul 13 20:19:14.166182 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.166165 2580 manager.go:349] Starting recovery of all containers Jul 13 20:19:14.167629 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.167603 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-52.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 13 20:19:14.167862 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.167840 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 13 20:19:14.168209 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.168189 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-5wc65" Jul 13 20:19:14.176139 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.175996 2580 manager.go:354] Recovery completed Jul 13 20:19:14.180027 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.180014 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.181634 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.181618 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.181686 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.181651 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.181686 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.181664 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.182278 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.182264 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.182505 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.182220 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 13 20:19:14.182567 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.182507 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 13 20:19:14.182567 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.182524 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 13 20:19:14.182988 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.182973 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.183063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.183003 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.183063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.183018 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.184555 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.184543 2580 policy_none.go:49] "None policy: Start" Jul 13 20:19:14.184603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.184559 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 13 20:19:14.184603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.184568 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 13 20:19:14.185274 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.185212 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-52.ec2.internal.18c1f2dc038b9ebd default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-52.ec2.internal,UID:ip-10-0-128-52.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-52.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-52.ec2.internal,},FirstTimestamp:2026-07-13 20:19:14.181635773 +0000 UTC m=+0.346195125,LastTimestamp:2026-07-13 20:19:14.181635773 +0000 UTC m=+0.346195125,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-52.ec2.internal,}" Jul 13 20:19:14.186538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.186527 2580 policy_none.go:47] "Start" Jul 13 20:19:14.230255 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230235 2580 manager.go:337] "Starting Device Plugin manager" Jul 13 20:19:14.230342 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.230266 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 13 20:19:14.230342 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230279 2580 server.go:87] "Starting device plugin registration server" Jul 13 20:19:14.230514 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230501 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 13 20:19:14.230565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230517 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 13 20:19:14.230630 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230591 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 13 20:19:14.230681 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230661 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 13 20:19:14.230681 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.230670 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 13 20:19:14.231131 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.231110 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 13 20:19:14.231223 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.231155 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.330948 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.330922 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.333036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.331805 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.333036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.331830 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.333036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.331843 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.333036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.331870 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.333036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.332702 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 13 20:19:14.334190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.334173 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 13 20:19:14.334285 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.334210 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 13 20:19:14.334285 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.334231 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 13 20:19:14.334285 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.334263 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 13 20:19:14.336671 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.336621 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:19:14.340811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.340793 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.340887 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.340812 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-52.ec2.internal\": node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.390680 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.390658 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.434782 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.434761 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal"] Jul 13 20:19:14.434842 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.434819 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.436364 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.436334 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.436437 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.436389 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.436437 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.436401 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.438702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.438689 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.438826 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.438810 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.438877 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.438849 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.441122 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441102 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.441190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441133 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.441190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441149 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.441190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441153 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.441190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441171 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.441190 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.441181 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.444013 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.444000 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.444056 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.444023 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 13 20:19:14.444775 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.444762 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientMemory" Jul 13 20:19:14.444819 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.444786 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasNoDiskPressure" Jul 13 20:19:14.444819 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.444797 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeHasSufficientPID" Jul 13 20:19:14.467344 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.467320 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-52.ec2.internal\" not found" node="ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.471667 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.471649 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-52.ec2.internal\" not found" node="ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.490754 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.490726 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.565296 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.565272 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.565372 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.565297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.565372 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.565314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7eb5df1a0ecc6549b309baa4fc43939-config\") pod \"kube-apiserver-proxy-ip-10-0-128-52.ec2.internal\" (UID: \"a7eb5df1a0ecc6549b309baa4fc43939\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.591404 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.591360 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.665926 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.665904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7eb5df1a0ecc6549b309baa4fc43939-config\") pod \"kube-apiserver-proxy-ip-10-0-128-52.ec2.internal\" (UID: \"a7eb5df1a0ecc6549b309baa4fc43939\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.665997 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.665928 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.665997 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.665944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.665997 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.665986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.666086 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.666006 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a7eb5df1a0ecc6549b309baa4fc43939-config\") pod \"kube-apiserver-proxy-ip-10-0-128-52.ec2.internal\" (UID: \"a7eb5df1a0ecc6549b309baa4fc43939\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.666086 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.666014 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/4d4d903d7d39d3e756d828f6662dd7a7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal\" (UID: \"4d4d903d7d39d3e756d828f6662dd7a7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.692039 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.692016 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.769546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.769514 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.775384 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:14.774665 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:14.792665 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.792640 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.893221 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.893148 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:14.994245 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:14.994213 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:15.094943 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:15.094913 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:15.107102 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.107079 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 13 20:19:15.107249 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.107229 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 13 20:19:15.174488 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.174418 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-12 20:14:15 +0000 UTC" deadline="2028-01-18 00:46:37.821851708 +0000 UTC" Jul 13 20:19:15.174488 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.174448 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13276h27m22.647408448s" Jul 13 20:19:15.195409 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:15.195384 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-128-52.ec2.internal\" not found" Jul 13 20:19:15.220240 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.220218 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:19:15.232974 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.232955 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:19:15.263711 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.263690 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" Jul 13 20:19:15.277782 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.277764 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:19:15.279499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.279485 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" Jul 13 20:19:15.286406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.286392 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 13 20:19:15.348000 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:15.347845 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7eb5df1a0ecc6549b309baa4fc43939.slice/crio-2699d041d910cdc2051771e173d2960ce2f406b6205d2fa3d7e228f15839b3f5 WatchSource:0}: Error finding container 2699d041d910cdc2051771e173d2960ce2f406b6205d2fa3d7e228f15839b3f5: Status 404 returned error can't find the container with id 2699d041d910cdc2051771e173d2960ce2f406b6205d2fa3d7e228f15839b3f5 Jul 13 20:19:15.348137 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:15.348121 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4d4d903d7d39d3e756d828f6662dd7a7.slice/crio-b10ab86e825b3d97ce3894e7631e824a276ad315cb55bc519b0d9da556564238 WatchSource:0}: Error finding container b10ab86e825b3d97ce3894e7631e824a276ad315cb55bc519b0d9da556564238: Status 404 returned error can't find the container with id b10ab86e825b3d97ce3894e7631e824a276ad315cb55bc519b0d9da556564238 Jul 13 20:19:15.352847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.352817 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:19:15.606156 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:15.606065 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:19:16.144590 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.144557 2580 apiserver.go:52] "Watching apiserver" Jul 13 20:19:16.153185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.153144 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 13 20:19:16.154313 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.154265 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-l25vp","openshift-image-registry/node-ca-9wtbh","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal","openshift-multus/multus-additional-cni-plugins-tw92n","openshift-multus/multus-zk9nn","openshift-network-diagnostics/network-check-target-ds8ss","openshift-network-operator/iptables-alerter-7ccwn","kube-system/konnectivity-agent-w58l4","kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76","openshift-dns/node-resolver-69s4w","openshift-multus/network-metrics-daemon-shvr5","openshift-ovn-kubernetes/ovnkube-node-4sksc","kube-system/global-pull-secret-syncer-p2ps2"] Jul 13 20:19:16.157389 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.157365 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.159861 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.159837 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.162021 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.161994 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.162139 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.162097 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:16.164499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.164478 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 13 20:19:16.164617 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.164600 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.167210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.167188 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.169732 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.169583 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:16.169732 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.169663 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:16.171402 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.170779 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 13 20:19:16.171402 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.171210 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-q47vc\"" Jul 13 20:19:16.172199 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.172178 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.174730 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.174710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.175226 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-os-release\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.175333 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-netns\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175333 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175290 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-ovn\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175452 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-netd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175452 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175361 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175452 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-config\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175452 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cni-binary-copy\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-slash\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175485 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-var-lib-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-log-socket\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175565 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-system-cni-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.175662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175636 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175660 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-k8s-cni-cncf-io\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-multus-certs\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175731 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0626783-392d-4d89-8c21-0e1f585e7fb5-host-slash\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175773 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-etc-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-socket-dir-parent\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175933 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-conf-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175969 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-etc-kubernetes\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.175996 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9sr7s\" (UniqueName: \"kubernetes.io/projected/2962d625-be0c-4c1e-ae90-f2577f6e5f20-kube-api-access-9sr7s\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-node-log\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-script-lib\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176074 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176098 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cnibin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-daemon-config\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176148 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfdf4\" (UniqueName: \"kubernetes.io/projected/d0626783-392d-4d89-8c21-0e1f585e7fb5-kube-api-access-dfdf4\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-systemd-units\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-bin\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-cnibin\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176261 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-85jkc\" (UniqueName: \"kubernetes.io/projected/81d105e4-a5b9-426c-988d-3988cf6b3706-kube-api-access-85jkc\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176294 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-system-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176322 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176346 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176377 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-env-overrides\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176400 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovn-node-metrics-cert\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.176415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176417 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xld6d\" (UniqueName: \"kubernetes.io/projected/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-kube-api-access-xld6d\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-bin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-multus\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176466 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-kubelet\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-hostroot\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-netns\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-dbus\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0626783-392d-4d89-8c21-0e1f585e7fb5-iptables-alerter-script\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-kubelet\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176711 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-systemd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-kubelet-config\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.176806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-os-release\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.177256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.177227 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.178766 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.178727 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tbzbj\"" Jul 13 20:19:16.179160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.179141 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-nxspx\"" Jul 13 20:19:16.179722 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.179707 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.180836 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.180356 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.180836 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.180551 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.180836 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.180742 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 13 20:19:16.182426 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.182406 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.182661 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.182646 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 13 20:19:16.182978 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.182962 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 13 20:19:16.183195 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.183181 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.184895 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.184874 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.184988 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.184944 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:16.185059 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.185050 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.187338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.187319 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.187437 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.187338 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 13 20:19:16.187720 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.187701 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.187835 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.187800 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mp72q\"" Jul 13 20:19:16.188011 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.187988 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 13 20:19:16.188332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.188309 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-x2nks\"" Jul 13 20:19:16.188945 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.188925 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.189748 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.189590 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.189984 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.189969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 13 20:19:16.190860 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.190833 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 13 20:19:16.191046 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.191026 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-k9xwq\"" Jul 13 20:19:16.192849 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.192430 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.192849 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.192494 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.194790 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.194190 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 13 20:19:16.194790 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.194386 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.194966 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.194918 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 13 20:19:16.196817 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.196799 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-t7d52\"" Jul 13 20:19:16.197560 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.197539 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 13 20:19:16.198055 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.198031 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 13 20:19:16.198250 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.198232 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.198523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.198502 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5k5qk\"" Jul 13 20:19:16.199718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.199700 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rckvq\"" Jul 13 20:19:16.200111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.200022 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 13 20:19:16.200266 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.200250 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 13 20:19:16.221778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.221731 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 13 20:19:16.226243 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.226221 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 13 20:19:16.244579 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.244560 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 13 20:19:16.265147 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.265128 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 13 20:19:16.270636 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.270615 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-kdgqs" Jul 13 20:19:16.277983 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.277960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-node-log\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.278083 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.277994 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-daemon-config\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278083 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.278083 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278051 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-var-lib-kubelet\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.278083 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9sr7s\" (UniqueName: \"kubernetes.io/projected/2962d625-be0c-4c1e-ae90-f2577f6e5f20-kube-api-access-9sr7s\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278094 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-node-log\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dfdf4\" (UniqueName: \"kubernetes.io/projected/d0626783-392d-4d89-8c21-0e1f585e7fb5-kube-api-access-dfdf4\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-bin\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278188 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-cnibin\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-85jkc\" (UniqueName: \"kubernetes.io/projected/81d105e4-a5b9-426c-988d-3988cf6b3706-kube-api-access-85jkc\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-kubelet\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-agent-certs\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278320 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovn-node-metrics-cert\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6nfhr\" (UniqueName: \"kubernetes.io/projected/58ec379f-96e0-41d3-a09e-bdc06bd97c08-kube-api-access-6nfhr\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278373 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-konnectivity-ca\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-sys\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278408 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-kubelet\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278459 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278418 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-host\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-bin\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rczg9\" (UniqueName: \"kubernetes.io/projected/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-kube-api-access-rczg9\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278576 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-cnibin\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-device-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278644 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-daemon-config\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.278687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9853f083-cbfb-439a-8698-4f85d7007f17-tmp-dir\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278710 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-dbus\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278769 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-dbus\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-systemd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278796 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278839 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-systemd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-os-release\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278879 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-os-release\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-os-release\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278934 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-socket-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-netns\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278973 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-os-release\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.278995 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-netd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279007 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-run-netns\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279047 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-cni-netd\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.279265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-config\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279080 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cni-binary-copy\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279123 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-system-cni-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279183 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-k8s-cni-cncf-io\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-multus-certs\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-etc-kubernetes\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-systemd\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9853f083-cbfb-439a-8698-4f85d7007f17-hosts-file\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279333 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0626783-392d-4d89-8c21-0e1f585e7fb5-host-slash\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279362 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-etc-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2h7t5\" (UniqueName: \"kubernetes.io/projected/c1146270-62c3-4f1e-b917-b3c45c509559-kube-api-access-2h7t5\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-script-lib\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-config\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279472 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cni-binary-copy\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.280081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cnibin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279517 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-k8s-cni-cncf-io\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279524 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-etc-kubernetes\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279517 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysconfig\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279552 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-system-cni-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279554 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-kubernetes\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279583 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-run\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279636 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-conf-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-systemd-units\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-system-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-bin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-multus\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-lib-modules\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279845 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1146270-62c3-4f1e-b917-b3c45c509559-host\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.279950 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovnkube-script-lib\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279978 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-multus-certs\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.281154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.279959 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-conf-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.280019 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:16.7799981 +0000 UTC m=+2.944557435 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-bin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280036 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-env-overrides\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280061 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d0626783-392d-4d89-8c21-0e1f585e7fb5-host-slash\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280070 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280056 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-var-lib-cni-multus\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280109 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xld6d\" (UniqueName: \"kubernetes.io/projected/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-kube-api-access-xld6d\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280136 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-system-cni-dir\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-hostroot\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280198 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-systemd-units\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280203 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-cnibin\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-env-overrides\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280310 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-etc-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-hostroot\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-registration-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdq7m\" (UniqueName: \"kubernetes.io/projected/9853f083-cbfb-439a-8698-4f85d7007f17-kube-api-access-jdq7m\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280465 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280490 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-netns\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280526 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0626783-392d-4d89-8c21-0e1f585e7fb5-iptables-alerter-script\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-kubelet\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280584 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-kubelet-config\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280589 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/81d105e4-a5b9-426c-988d-3988cf6b3706-tuning-conf-dir\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1146270-62c3-4f1e-b917-b3c45c509559-serviceca\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280631 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-sys-fs\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280645 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0bd6ef49-c50c-44bc-b547-95958e3de5bb-kubelet-config\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280655 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-kubelet\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280659 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-ovn\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280701 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-run-ovn\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280703 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-host-run-netns\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.282912 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280737 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-modprobe-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280787 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-conf\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-tuned\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-tmp\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280861 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-slash\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-var-lib-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-log-socket\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280942 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-host-slash\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-var-lib-openvswitch\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280986 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.280986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-log-socket\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281016 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsnfq\" (UniqueName: \"kubernetes.io/projected/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kube-api-access-lsnfq\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281054 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-socket-dir-parent\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281130 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/2962d625-be0c-4c1e-ae90-f2577f6e5f20-multus-socket-dir-parent\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281216 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.283523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281832 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d0626783-392d-4d89-8c21-0e1f585e7fb5-iptables-alerter-script\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.284143 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.281986 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/81d105e4-a5b9-426c-988d-3988cf6b3706-cni-binary-copy\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.284143 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.283258 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-ovn-node-metrics-cert\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.296920 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.296900 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-kdgqs" Jul 13 20:19:16.308866 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.308839 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9sr7s\" (UniqueName: \"kubernetes.io/projected/2962d625-be0c-4c1e-ae90-f2577f6e5f20-kube-api-access-9sr7s\") pod \"multus-zk9nn\" (UID: \"2962d625-be0c-4c1e-ae90-f2577f6e5f20\") " pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.312223 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.312200 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xld6d\" (UniqueName: \"kubernetes.io/projected/f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f-kube-api-access-xld6d\") pod \"ovnkube-node-4sksc\" (UID: \"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f\") " pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.318796 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.318753 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-85jkc\" (UniqueName: \"kubernetes.io/projected/81d105e4-a5b9-426c-988d-3988cf6b3706-kube-api-access-85jkc\") pod \"multus-additional-cni-plugins-tw92n\" (UID: \"81d105e4-a5b9-426c-988d-3988cf6b3706\") " pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.321527 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.321504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfdf4\" (UniqueName: \"kubernetes.io/projected/d0626783-392d-4d89-8c21-0e1f585e7fb5-kube-api-access-dfdf4\") pod \"iptables-alerter-7ccwn\" (UID: \"d0626783-392d-4d89-8c21-0e1f585e7fb5\") " pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.338380 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.338331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" event={"ID":"4d4d903d7d39d3e756d828f6662dd7a7","Type":"ContainerStarted","Data":"b10ab86e825b3d97ce3894e7631e824a276ad315cb55bc519b0d9da556564238"} Jul 13 20:19:16.339583 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.339555 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" event={"ID":"a7eb5df1a0ecc6549b309baa4fc43939","Type":"ContainerStarted","Data":"2699d041d910cdc2051771e173d2960ce2f406b6205d2fa3d7e228f15839b3f5"} Jul 13 20:19:16.382270 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-device-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9853f083-cbfb-439a-8698-4f85d7007f17-tmp-dir\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-socket-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-systemd\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382329 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-device-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9853f083-cbfb-439a-8698-4f85d7007f17-hosts-file\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382358 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2h7t5\" (UniqueName: \"kubernetes.io/projected/c1146270-62c3-4f1e-b917-b3c45c509559-kube-api-access-2h7t5\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.382401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-systemd\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382433 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/9853f083-cbfb-439a-8698-4f85d7007f17-hosts-file\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382469 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysconfig\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382495 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-kubernetes\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382511 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-socket-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382517 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-run\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382556 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-run\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382579 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-lib-modules\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysconfig\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382627 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1146270-62c3-4f1e-b917-b3c45c509559-host\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382647 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-kubernetes\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382680 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c1146270-62c3-4f1e-b917-b3c45c509559-host\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-registration-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382705 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/9853f083-cbfb-439a-8698-4f85d7007f17-tmp-dir\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382731 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-lib-modules\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdq7m\" (UniqueName: \"kubernetes.io/projected/9853f083-cbfb-439a-8698-4f85d7007f17-kube-api-access-jdq7m\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382788 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-registration-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.382845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382824 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1146270-62c3-4f1e-b917-b3c45c509559-serviceca\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-sys-fs\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-modprobe-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382970 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-conf\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-sys-fs\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.382937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kubelet-dir\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383008 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-tuned\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-tmp\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-modprobe-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383079 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383110 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsnfq\" (UniqueName: \"kubernetes.io/projected/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kube-api-access-lsnfq\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383193 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-var-lib-kubelet\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-agent-certs\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383253 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6nfhr\" (UniqueName: \"kubernetes.io/projected/58ec379f-96e0-41d3-a09e-bdc06bd97c08-kube-api-access-6nfhr\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.384069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383273 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/d3dbd287-4251-47f1-af72-5e8896d2b7fb-etc-selinux\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383279 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-konnectivity-ca\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-sys\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383193 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-conf\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c1146270-62c3-4f1e-b917-b3c45c509559-serviceca\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383330 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-host\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-host\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rczg9\" (UniqueName: \"kubernetes.io/projected/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-kube-api-access-rczg9\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383649 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-sysctl-d\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.383697 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383703 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-var-lib-kubelet\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.383748 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:16.883734187 +0000 UTC m=+3.048293517 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.383999 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-sys\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.385084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.384483 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-konnectivity-ca\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.385917 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.385886 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-tmp\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.386156 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.386132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-etc-tuned\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.386236 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.386221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/fb761904-bc7d-42fa-bc30-b4b5dfbc188e-agent-certs\") pod \"konnectivity-agent-w58l4\" (UID: \"fb761904-bc7d-42fa-bc30-b4b5dfbc188e\") " pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.408227 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.408174 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6nfhr\" (UniqueName: \"kubernetes.io/projected/58ec379f-96e0-41d3-a09e-bdc06bd97c08-kube-api-access-6nfhr\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.408564 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.408541 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsnfq\" (UniqueName: \"kubernetes.io/projected/d3dbd287-4251-47f1-af72-5e8896d2b7fb-kube-api-access-lsnfq\") pod \"aws-ebs-csi-driver-node-kxk76\" (UID: \"d3dbd287-4251-47f1-af72-5e8896d2b7fb\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.408796 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.408776 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:16.408878 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.408803 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:16.408878 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.408836 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:16.408957 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.408906 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:16.908885161 +0000 UTC m=+3.073444492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:16.410010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.409984 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2h7t5\" (UniqueName: \"kubernetes.io/projected/c1146270-62c3-4f1e-b917-b3c45c509559-kube-api-access-2h7t5\") pod \"node-ca-9wtbh\" (UID: \"c1146270-62c3-4f1e-b917-b3c45c509559\") " pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.411123 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.411102 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdq7m\" (UniqueName: \"kubernetes.io/projected/9853f083-cbfb-439a-8698-4f85d7007f17-kube-api-access-jdq7m\") pod \"node-resolver-69s4w\" (UID: \"9853f083-cbfb-439a-8698-4f85d7007f17\") " pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.413570 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.413551 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rczg9\" (UniqueName: \"kubernetes.io/projected/b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90-kube-api-access-rczg9\") pod \"tuned-l25vp\" (UID: \"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90\") " pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.472159 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.472123 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7ccwn" Jul 13 20:19:16.481945 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.481923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:16.499082 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.499059 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:16.505947 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.505923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-tw92n" Jul 13 20:19:16.512846 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.512819 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-zk9nn" Jul 13 20:19:16.521883 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.521862 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-l25vp" Jul 13 20:19:16.531777 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.531759 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-9wtbh" Jul 13 20:19:16.539809 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.539793 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" Jul 13 20:19:16.544617 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.544597 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-69s4w" Jul 13 20:19:16.786172 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.786085 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:16.786344 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.786287 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:16.786409 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.786361 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:17.786343348 +0000 UTC m=+3.950902683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:16.887174 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.887137 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:16.887357 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.887275 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:16.887357 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.887345 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:17.887325867 +0000 UTC m=+4.051885198 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:16.987982 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:16.987943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:16.988150 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.988129 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:16.988193 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.988157 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:16.988193 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.988167 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:16.988272 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:16.988218 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:17.988204576 +0000 UTC m=+4.152763906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:17.302074 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.302004 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:14:17 +0000 UTC" deadline="2028-04-23 08:25:53.123556082 +0000 UTC" Jul 13 20:19:17.302074 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.302066 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15588h6m35.821496014s" Jul 13 20:19:17.335331 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.335302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:17.335467 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.335406 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:17.384577 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.384552 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3dbd287_4251_47f1_af72_5e8896d2b7fb.slice/crio-54b3bb4ef8bbe58b7cbf240e91d2f921a16ca9b103f026e9b36e3e5e3d7f55cb WatchSource:0}: Error finding container 54b3bb4ef8bbe58b7cbf240e91d2f921a16ca9b103f026e9b36e3e5e3d7f55cb: Status 404 returned error can't find the container with id 54b3bb4ef8bbe58b7cbf240e91d2f921a16ca9b103f026e9b36e3e5e3d7f55cb Jul 13 20:19:17.395370 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.395340 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2962d625_be0c_4c1e_ae90_f2577f6e5f20.slice/crio-4643305869aaecbeba058af1e9d1407c5ca322ed632d19914a37e9b0c4266c7c WatchSource:0}: Error finding container 4643305869aaecbeba058af1e9d1407c5ca322ed632d19914a37e9b0c4266c7c: Status 404 returned error can't find the container with id 4643305869aaecbeba058af1e9d1407c5ca322ed632d19914a37e9b0c4266c7c Jul 13 20:19:17.397520 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.397495 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd0626783_392d_4d89_8c21_0e1f585e7fb5.slice/crio-122dc98ce7f8bdd71878a732bc458cdbc26b0ea1e7f124f4761cbb3fc16727b0 WatchSource:0}: Error finding container 122dc98ce7f8bdd71878a732bc458cdbc26b0ea1e7f124f4761cbb3fc16727b0: Status 404 returned error can't find the container with id 122dc98ce7f8bdd71878a732bc458cdbc26b0ea1e7f124f4761cbb3fc16727b0 Jul 13 20:19:17.398893 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.398869 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf63aee13_3d39_4eeb_b7c5_eecd7fe21d0f.slice/crio-a55bd1416f2bf3950e485eb8914b85897a2cd963c42bff24b733a65cdb6fc6fb WatchSource:0}: Error finding container a55bd1416f2bf3950e485eb8914b85897a2cd963c42bff24b733a65cdb6fc6fb: Status 404 returned error can't find the container with id a55bd1416f2bf3950e485eb8914b85897a2cd963c42bff24b733a65cdb6fc6fb Jul 13 20:19:17.400347 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.400328 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb20e3596_ac0d_4fa6_9c6c_3b4eb6a49b90.slice/crio-b7713d7f2cb7c23a1b6b56c92128e25806b147e4a831806606b69d54e321c477 WatchSource:0}: Error finding container b7713d7f2cb7c23a1b6b56c92128e25806b147e4a831806606b69d54e321c477: Status 404 returned error can't find the container with id b7713d7f2cb7c23a1b6b56c92128e25806b147e4a831806606b69d54e321c477 Jul 13 20:19:17.401878 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.401847 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1146270_62c3_4f1e_b917_b3c45c509559.slice/crio-4e04b85f880cd6ff156aa0d55da0a0f31e8e90fc0c007d5f76b1c26f168b368e WatchSource:0}: Error finding container 4e04b85f880cd6ff156aa0d55da0a0f31e8e90fc0c007d5f76b1c26f168b368e: Status 404 returned error can't find the container with id 4e04b85f880cd6ff156aa0d55da0a0f31e8e90fc0c007d5f76b1c26f168b368e Jul 13 20:19:17.403343 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.403276 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9853f083_cbfb_439a_8698_4f85d7007f17.slice/crio-f93221ec1b621e233ef1e891de174f0210933def3f6d43a193a2120dc0dfb3c5 WatchSource:0}: Error finding container f93221ec1b621e233ef1e891de174f0210933def3f6d43a193a2120dc0dfb3c5: Status 404 returned error can't find the container with id f93221ec1b621e233ef1e891de174f0210933def3f6d43a193a2120dc0dfb3c5 Jul 13 20:19:17.404460 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.404440 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb761904_bc7d_42fa_bc30_b4b5dfbc188e.slice/crio-a963e790522e89e707d8a9cd444ee8e4f61dcd0f84542fd5c19310502d3621c7 WatchSource:0}: Error finding container a963e790522e89e707d8a9cd444ee8e4f61dcd0f84542fd5c19310502d3621c7: Status 404 returned error can't find the container with id a963e790522e89e707d8a9cd444ee8e4f61dcd0f84542fd5c19310502d3621c7 Jul 13 20:19:17.405233 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:17.405215 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81d105e4_a5b9_426c_988d_3988cf6b3706.slice/crio-3e88ec9388e995b0a6d17af27cf2f418cc7104bb02a0fc6a7bf90dfd00b6f027 WatchSource:0}: Error finding container 3e88ec9388e995b0a6d17af27cf2f418cc7104bb02a0fc6a7bf90dfd00b6f027: Status 404 returned error can't find the container with id 3e88ec9388e995b0a6d17af27cf2f418cc7104bb02a0fc6a7bf90dfd00b6f027 Jul 13 20:19:17.795183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.794903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:17.795360 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.795058 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:17.795937 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.795521 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:19.795494542 +0000 UTC m=+5.960053887 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:17.896465 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.896417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:17.896651 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.896619 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:17.896735 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.896706 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:19.896684189 +0000 UTC m=+6.061243520 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:17.997450 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:17.997417 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:17.997639 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.997560 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:17.997639 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.997574 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:17.997639 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.997584 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:17.997639 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:17.997626 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:19.997613374 +0000 UTC m=+6.162172707 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:18.303309 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.303243 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-12 20:14:17 +0000 UTC" deadline="2028-03-02 14:00:20.444283009 +0000 UTC" Jul 13 20:19:18.303309 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.303309 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14345h41m2.14097835s" Jul 13 20:19:18.336388 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.335955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:18.336388 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:18.336063 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:18.336388 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.336172 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:18.336388 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:18.336311 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:18.349695 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.349662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-69s4w" event={"ID":"9853f083-cbfb-439a-8698-4f85d7007f17","Type":"ContainerStarted","Data":"f93221ec1b621e233ef1e891de174f0210933def3f6d43a193a2120dc0dfb3c5"} Jul 13 20:19:18.355736 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.355708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ccwn" event={"ID":"d0626783-392d-4d89-8c21-0e1f585e7fb5","Type":"ContainerStarted","Data":"122dc98ce7f8bdd71878a732bc458cdbc26b0ea1e7f124f4761cbb3fc16727b0"} Jul 13 20:19:18.359956 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.359928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zk9nn" event={"ID":"2962d625-be0c-4c1e-ae90-f2577f6e5f20","Type":"ContainerStarted","Data":"4643305869aaecbeba058af1e9d1407c5ca322ed632d19914a37e9b0c4266c7c"} Jul 13 20:19:18.365904 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.365830 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" event={"ID":"d3dbd287-4251-47f1-af72-5e8896d2b7fb","Type":"ContainerStarted","Data":"54b3bb4ef8bbe58b7cbf240e91d2f921a16ca9b103f026e9b36e3e5e3d7f55cb"} Jul 13 20:19:18.376850 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.376754 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" event={"ID":"a7eb5df1a0ecc6549b309baa4fc43939","Type":"ContainerStarted","Data":"9aa2f9d59d71c5cf97f1e16b34e230fdaab75c860ef4f389e94bd6b812ae0229"} Jul 13 20:19:18.390088 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.390057 2580 generic.go:356] "Generic (PLEG): container finished" podID="4d4d903d7d39d3e756d828f6662dd7a7" containerID="dcc43a6c13728256dd2d72483bda24e611c89dadc9505a874293a77d3416f8c7" exitCode=0 Jul 13 20:19:18.390254 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.390139 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" event={"ID":"4d4d903d7d39d3e756d828f6662dd7a7","Type":"ContainerDied","Data":"dcc43a6c13728256dd2d72483bda24e611c89dadc9505a874293a77d3416f8c7"} Jul 13 20:19:18.393260 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.393195 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-52.ec2.internal" podStartSLOduration=2.393177639 podStartE2EDuration="2.393177639s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:18.391937317 +0000 UTC m=+4.556496673" watchObservedRunningTime="2026-07-13 20:19:18.393177639 +0000 UTC m=+4.557736996" Jul 13 20:19:18.398791 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.398760 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9wtbh" event={"ID":"c1146270-62c3-4f1e-b917-b3c45c509559","Type":"ContainerStarted","Data":"4e04b85f880cd6ff156aa0d55da0a0f31e8e90fc0c007d5f76b1c26f168b368e"} Jul 13 20:19:18.423070 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.422975 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l25vp" event={"ID":"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90","Type":"ContainerStarted","Data":"b7713d7f2cb7c23a1b6b56c92128e25806b147e4a831806606b69d54e321c477"} Jul 13 20:19:18.426587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.426557 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerStarted","Data":"3e88ec9388e995b0a6d17af27cf2f418cc7104bb02a0fc6a7bf90dfd00b6f027"} Jul 13 20:19:18.435159 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.435129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"a55bd1416f2bf3950e485eb8914b85897a2cd963c42bff24b733a65cdb6fc6fb"} Jul 13 20:19:18.443486 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:18.443454 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w58l4" event={"ID":"fb761904-bc7d-42fa-bc30-b4b5dfbc188e","Type":"ContainerStarted","Data":"a963e790522e89e707d8a9cd444ee8e4f61dcd0f84542fd5c19310502d3621c7"} Jul 13 20:19:19.335105 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:19.334487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:19.335105 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:19.334643 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:19.482377 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:19.482339 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" event={"ID":"4d4d903d7d39d3e756d828f6662dd7a7","Type":"ContainerStarted","Data":"75f509003cf7404dc5893dfe425315803484e5e8b1719ac2ca93eb46c9414a61"} Jul 13 20:19:19.814349 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:19.814258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:19.814520 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:19.814411 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:19.814520 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:19.814506 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:23.814482485 +0000 UTC m=+9.979041821 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:19.915325 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:19.914605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:19.915325 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:19.914855 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:19.915325 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:19.914925 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:23.91490439 +0000 UTC m=+10.079463726 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:20.015526 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:20.015488 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:20.015728 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.015707 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:20.015848 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.015739 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:20.015848 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.015753 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:20.015848 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.015840 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:24.015817535 +0000 UTC m=+10.180376868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:20.337411 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:20.337371 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:20.338012 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.337575 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:20.338012 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:20.337989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:20.338153 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:20.338098 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:21.335138 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:21.335097 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:21.335329 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:21.335237 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:22.337879 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:22.337182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:22.337879 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:22.337331 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:22.337879 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:22.337672 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:22.337879 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:22.337742 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:23.335282 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:23.335242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:23.335481 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:23.335396 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:23.850350 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:23.850308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:23.850769 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:23.850488 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:23.850769 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:23.850583 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:31.850560123 +0000 UTC m=+18.015119479 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:23.951377 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:23.951336 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:23.951592 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:23.951567 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:23.951651 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:23.951639 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:31.951618314 +0000 UTC m=+18.116177649 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:24.052164 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:24.052124 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:24.052333 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.052317 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:24.052391 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.052337 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:24.052391 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.052352 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:24.052500 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.052412 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:32.052392084 +0000 UTC m=+18.216951437 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:24.336084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:24.335992 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:24.336241 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.336111 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:24.336380 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:24.336352 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:24.336491 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:24.336471 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:25.335484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:25.335445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:25.335990 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:25.335599 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:26.336627 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:26.335964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:26.336627 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:26.336106 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:26.336627 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:26.335975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:26.336627 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:26.336579 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:27.334929 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:27.334894 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:27.335118 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:27.335040 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:28.335068 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:28.335036 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:28.335489 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:28.335161 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:28.335612 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:28.335592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:28.335714 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:28.335693 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:29.334698 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:29.334660 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:29.334903 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:29.334813 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:30.335526 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:30.335483 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:30.336033 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:30.335622 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:30.336033 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:30.335679 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:30.336033 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:30.335826 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:31.335406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:31.335376 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:31.335579 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:31.335511 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:31.920341 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:31.920306 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:31.920544 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:31.920467 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:31.920544 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:31.920541 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:19:47.920521419 +0000 UTC m=+34.085080748 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:32.021553 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:32.021516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:32.021707 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.021639 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:32.021707 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.021698 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:48.021678448 +0000 UTC m=+34.186237782 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:32.122368 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:32.122335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:32.122554 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.122533 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:32.122597 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.122566 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:32.122597 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.122580 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:32.122671 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.122642 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:19:48.122623841 +0000 UTC m=+34.287183183 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:32.334721 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:32.334647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:32.334880 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.334787 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:32.334880 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:32.334853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:32.334989 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:32.334942 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:33.334510 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:33.334478 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:33.334913 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:33.334601 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:34.336071 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:34.335640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:34.336071 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:34.335728 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:34.336071 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:34.335734 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:34.336071 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:34.335861 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:35.335182 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:35.335142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:35.335378 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:35.335260 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:36.335234 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:36.335194 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:36.335234 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:36.335227 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:36.335979 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:36.335317 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:36.335979 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:36.335449 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:37.335125 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.334475 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:37.335125 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:37.334772 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:37.518723 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.518690 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-9wtbh" event={"ID":"c1146270-62c3-4f1e-b917-b3c45c509559","Type":"ContainerStarted","Data":"37a9dfaf63ac869f7616112974f200d2668d712d0bca79a9d336ac3ef62a398d"} Jul 13 20:19:37.520464 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.520438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-l25vp" event={"ID":"b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90","Type":"ContainerStarted","Data":"6fb8aba167b5246c8e7be6f37334fc42e2d1709469bb112691641befe1df2b1f"} Jul 13 20:19:37.522221 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.522167 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerStarted","Data":"155501e16d6fa377a63a67cef054a727cd42fc53e7fff82af1110a85f1761100"} Jul 13 20:19:37.523890 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.523860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"a24cd6da817b9108104b123aa675fc16d5087605ed031b9e316cf9eb8c646054"} Jul 13 20:19:37.526800 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.526773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-w58l4" event={"ID":"fb761904-bc7d-42fa-bc30-b4b5dfbc188e","Type":"ContainerStarted","Data":"6f77f78562331056ba4fd2022624ed3a3a7f14d0e2d6dc55ddbd0896ae4680fa"} Jul 13 20:19:37.528307 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.528272 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-69s4w" event={"ID":"9853f083-cbfb-439a-8698-4f85d7007f17","Type":"ContainerStarted","Data":"8b9c213ff3de33bad3c75a4ebda8734c55d0df629adb3257439880126cd3bdbd"} Jul 13 20:19:37.530043 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.530015 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-zk9nn" event={"ID":"2962d625-be0c-4c1e-ae90-f2577f6e5f20","Type":"ContainerStarted","Data":"9c0f15b69abe78557ad77c1f1fc5239f05059ebc9461ad5d5ad9954e2cbf057d"} Jul 13 20:19:37.531487 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.531437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" event={"ID":"d3dbd287-4251-47f1-af72-5e8896d2b7fb","Type":"ContainerStarted","Data":"be83425edc9e7b33b334b3072dc09eab9e4b138525f41e3952ce9133b6a43ef2"} Jul 13 20:19:37.533832 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.533782 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-9wtbh" podStartSLOduration=2.7670153109999998 podStartE2EDuration="22.533765619s" totalImagesPullingTime="19.766750308s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:37.533496377 +0000 UTC m=+23.698055772" watchObservedRunningTime="2026-07-13 20:19:37.533765619 +0000 UTC m=+23.698324970" Jul 13 20:19:37.533932 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.533887 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-52.ec2.internal" podStartSLOduration=21.533876741 podStartE2EDuration="21.533876741s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:19.501084824 +0000 UTC m=+5.665644181" watchObservedRunningTime="2026-07-13 20:19:37.533876741 +0000 UTC m=+23.698436093" Jul 13 20:19:37.547832 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.547775 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-w58l4" podStartSLOduration=7.5923606150000005 podStartE2EDuration="22.547735756s" totalImagesPullingTime="14.955375141s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:37.547232862 +0000 UTC m=+23.711792212" watchObservedRunningTime="2026-07-13 20:19:37.547735756 +0000 UTC m=+23.712295108" Jul 13 20:19:37.586476 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.586435 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-zk9nn" podStartSLOduration=2.789987828 podStartE2EDuration="22.586420671s" totalImagesPullingTime="19.796432843s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:37.58594654 +0000 UTC m=+23.750505892" watchObservedRunningTime="2026-07-13 20:19:37.586420671 +0000 UTC m=+23.750980021" Jul 13 20:19:37.600470 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.600428 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-l25vp" podStartSLOduration=2.829332421 podStartE2EDuration="22.600411161s" totalImagesPullingTime="19.77107874s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:37.60017231 +0000 UTC m=+23.764731661" watchObservedRunningTime="2026-07-13 20:19:37.600411161 +0000 UTC m=+23.764970514" Jul 13 20:19:37.614283 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:37.614242 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-69s4w" podStartSLOduration=2.871052369 podStartE2EDuration="22.614223322s" totalImagesPullingTime="19.743170953s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:37.613918754 +0000 UTC m=+23.778478107" watchObservedRunningTime="2026-07-13 20:19:37.614223322 +0000 UTC m=+23.778782677" Jul 13 20:19:38.334615 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.334463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:38.334693 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.334519 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:38.334728 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:38.334696 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:38.334851 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:38.334783 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:38.409718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.409696 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 13 20:19:38.535480 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.535401 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="155501e16d6fa377a63a67cef054a727cd42fc53e7fff82af1110a85f1761100" exitCode=0 Jul 13 20:19:38.535984 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.535489 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"155501e16d6fa377a63a67cef054a727cd42fc53e7fff82af1110a85f1761100"} Jul 13 20:19:38.538556 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.538531 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:19:38.538965 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.538943 2580 generic.go:356] "Generic (PLEG): container finished" podID="f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f" containerID="a97fdd291dfecbbbdb76aca352ffabe7b8af13c1a01881b24e01dee04c3496e4" exitCode=1 Jul 13 20:19:38.539072 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.539027 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"176985faba327abdb2ff49495db6f99308ec5718a55231790234cbe7d6dd6a28"} Jul 13 20:19:38.539133 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.539107 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"aa9bd5ca0b4867c6c425f23fbf5b988e7ab3cf87325f0fa9f1701c6d35ec11da"} Jul 13 20:19:38.539188 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.539156 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"ed0771eeb246d082e6afd755094be346d382aba8ec59d448b9b96f9af5859512"} Jul 13 20:19:38.539242 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.539207 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"416622f1f3465ed86bdda7888052b4c6808cbc03e2ce211ae9e8b3946b02f955"} Jul 13 20:19:38.539295 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.539257 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerDied","Data":"a97fdd291dfecbbbdb76aca352ffabe7b8af13c1a01881b24e01dee04c3496e4"} Jul 13 20:19:38.540427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.540408 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7ccwn" event={"ID":"d0626783-392d-4d89-8c21-0e1f585e7fb5","Type":"ContainerStarted","Data":"23a0b48a4a5d949133289b8a8d53d961358ea3bd1bd9dd4257c4e5d759c0134b"} Jul 13 20:19:38.542091 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.542050 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" event={"ID":"d3dbd287-4251-47f1-af72-5e8896d2b7fb","Type":"ContainerStarted","Data":"fcafddd90d08620d12ad115d486f86d192084d36b10be140f67ba9d07cfc6574"} Jul 13 20:19:38.573540 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:38.573504 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7ccwn" podStartSLOduration=3.801417528 podStartE2EDuration="23.573492783s" totalImagesPullingTime="19.772075255s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:38.573085341 +0000 UTC m=+24.737644691" watchObservedRunningTime="2026-07-13 20:19:38.573492783 +0000 UTC m=+24.738052134" Jul 13 20:19:39.203955 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.203924 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:39.204529 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.204507 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:39.245472 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.245392 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-13T20:19:38.409713111Z","UUID":"a2810352-8e88-434e-bffb-4e5e4452690a","Handler":null,"Name":"","Endpoint":""} Jul 13 20:19:39.248092 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.248071 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 13 20:19:39.248186 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.248116 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 13 20:19:39.334792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.334761 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:39.334957 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:39.334875 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:39.546701 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.546625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" event={"ID":"d3dbd287-4251-47f1-af72-5e8896d2b7fb","Type":"ContainerStarted","Data":"8e1d4a6236280b87493f20fcd9123f93a31ce075710d245cac2a6ef9f147f47b"} Jul 13 20:19:39.574752 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:39.574675 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-kxk76" podStartSLOduration=3.1403758330000002 podStartE2EDuration="24.574660377s" totalImagesPullingTime="21.434284544s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:39.574308028 +0000 UTC m=+25.738867380" watchObservedRunningTime="2026-07-13 20:19:39.574660377 +0000 UTC m=+25.739219728" Jul 13 20:19:40.335684 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:40.335425 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:40.335863 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:40.335499 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:40.335863 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:40.335711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:40.335983 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:40.335872 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:40.552608 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:40.552578 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:19:40.553321 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:40.553298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"ca79464a71b15fa89c4b0eebd7e4c2f259e3c0eb1657beca1a0528491992617c"} Jul 13 20:19:40.553457 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:40.553447 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:19:41.334954 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:41.334922 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:41.335139 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:41.335031 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:42.335343 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:42.335261 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:42.335940 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:42.335401 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:42.335940 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:42.335445 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:42.335940 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:42.335564 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:42.442482 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:42.442444 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:42.442649 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:42.442567 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 13 20:19:42.443018 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:42.442994 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-w58l4" Jul 13 20:19:43.335178 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.334954 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:43.335329 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:43.335210 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:43.562333 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.562296 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="4dd2a07019701855d282373b8a621f87c6f2090a39187c999ba52e8eb4dfc9ad" exitCode=0 Jul 13 20:19:43.562938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.562354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"4dd2a07019701855d282373b8a621f87c6f2090a39187c999ba52e8eb4dfc9ad"} Jul 13 20:19:43.566031 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.566017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:19:43.566373 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.566354 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"81f0c3befef1d0263c5d8b73abd9584d7e49041d6eed8a265de9644bcc5957a9"} Jul 13 20:19:43.566585 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.566572 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:43.566628 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.566615 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:43.566768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.566753 2580 scope.go:117] "RemoveContainer" containerID="a97fdd291dfecbbbdb76aca352ffabe7b8af13c1a01881b24e01dee04c3496e4" Jul 13 20:19:43.584221 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:43.584199 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:44.335415 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.335382 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:44.335598 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:44.335475 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:44.335769 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.335727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:44.335890 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:44.335868 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:44.570356 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.570161 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="a0f586354102eeb39a022153ce9abc1753bfab1a6d0fcda4f1efd035c9728e52" exitCode=0 Jul 13 20:19:44.570737 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.570232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"a0f586354102eeb39a022153ce9abc1753bfab1a6d0fcda4f1efd035c9728e52"} Jul 13 20:19:44.577095 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.577078 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:19:44.577557 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.577534 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" event={"ID":"f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f","Type":"ContainerStarted","Data":"6566f8632e0369eb94f46f8b2325cad0f4dcaace7be3bcaed8b9f59ff37200f4"} Jul 13 20:19:44.577726 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.577712 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:44.593009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.592949 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:19:44.631106 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.631056 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" podStartSLOduration=9.806958111 podStartE2EDuration="29.631039733s" totalImagesPullingTime="19.824081622s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:44.630138334 +0000 UTC m=+30.794697685" watchObservedRunningTime="2026-07-13 20:19:44.631039733 +0000 UTC m=+30.795599084" Jul 13 20:19:44.672250 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.672226 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p2ps2"] Jul 13 20:19:44.672402 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.672340 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:44.672456 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:44.672441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:44.676829 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.676808 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ds8ss"] Jul 13 20:19:44.676946 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.676902 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:44.677018 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:44.677001 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:44.680547 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.680528 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-shvr5"] Jul 13 20:19:44.680624 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:44.680617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:44.680728 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:44.680704 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:45.582160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:45.582066 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="cfbdc3c538c7a4ac3840ec09a69c9af0aa7c274b02093772800623b8ca214280" exitCode=0 Jul 13 20:19:45.582160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:45.582140 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"cfbdc3c538c7a4ac3840ec09a69c9af0aa7c274b02093772800623b8ca214280"} Jul 13 20:19:46.335577 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:46.335543 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:46.335735 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:46.335610 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:46.335735 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:46.335625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:46.335855 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:46.335759 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:46.335931 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:46.335834 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:46.335982 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:46.335944 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:47.941510 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:47.941474 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:47.942300 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:47.941606 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:47.942300 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:47.941672 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret podName:0bd6ef49-c50c-44bc-b547-95958e3de5bb nodeName:}" failed. No retries permitted until 2026-07-13 20:20:19.941652255 +0000 UTC m=+66.106211589 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret") pod "global-pull-secret-syncer-p2ps2" (UID: "0bd6ef49-c50c-44bc-b547-95958e3de5bb") : object "kube-system"/"original-pull-secret" not registered Jul 13 20:19:48.042548 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:48.042511 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:48.042711 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.042671 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:48.042799 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.042735 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs podName:58ec379f-96e0-41d3-a09e-bdc06bd97c08 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:20.042714554 +0000 UTC m=+66.207273883 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs") pod "network-metrics-daemon-shvr5" (UID: "58ec379f-96e0-41d3-a09e-bdc06bd97c08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 13 20:19:48.143968 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:48.143930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:48.144161 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.144092 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 13 20:19:48.144161 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.144110 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 13 20:19:48.144161 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.144120 2580 projected.go:196] Error preparing data for projected volume kube-api-access-4b4mx for pod openshift-network-diagnostics/network-check-target-ds8ss: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:48.144320 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.144170 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx podName:58065293-dd9f-4398-8e33-f48831027fda nodeName:}" failed. No retries permitted until 2026-07-13 20:20:20.144157325 +0000 UTC m=+66.308716654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-4b4mx" (UniqueName: "kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx") pod "network-check-target-ds8ss" (UID: "58065293-dd9f-4398-8e33-f48831027fda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:48.334486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:48.334523 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:48.334493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.334614 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.334701 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:48.335010 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:48.334779 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:50.334861 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:50.334806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:50.334861 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:50.334824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:50.335276 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:50.334811 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:50.335276 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:50.334904 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-p2ps2" podUID="0bd6ef49-c50c-44bc-b547-95958e3de5bb" Jul 13 20:19:50.335276 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:50.334988 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-ds8ss" podUID="58065293-dd9f-4398-8e33-f48831027fda" Jul 13 20:19:50.335276 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:50.335078 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-shvr5" podUID="58ec379f-96e0-41d3-a09e-bdc06bd97c08" Jul 13 20:19:50.597126 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:50.597044 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="a319e17be655a361995798c0a7c679ee249adf2ad8aff0ccbd230e59e46645b0" exitCode=0 Jul 13 20:19:50.597126 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:50.597108 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"a319e17be655a361995798c0a7c679ee249adf2ad8aff0ccbd230e59e46645b0"} Jul 13 20:19:51.199220 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.199190 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-128-52.ec2.internal" event="NodeReady" Jul 13 20:19:51.199388 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.199314 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 13 20:19:51.239927 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.239896 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-7fd4cb9555-wlrqt"] Jul 13 20:19:51.263682 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.263657 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-7fd4cb9555-wlrqt"] Jul 13 20:19:51.263839 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.263693 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-hrhss"] Jul 13 20:19:51.263839 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.263800 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.266614 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.266589 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 13 20:19:51.266704 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.266621 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 13 20:19:51.266777 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.266761 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-82c5x\"" Jul 13 20:19:51.266946 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.266931 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 13 20:19:51.273142 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.273124 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 13 20:19:51.285807 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.285788 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-4c2pr"] Jul 13 20:19:51.285921 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.285905 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.287905 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.287887 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 13 20:19:51.287986 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.287963 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 13 20:19:51.288029 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.288005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t5qs6\"" Jul 13 20:19:51.309706 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.309686 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4c2pr"] Jul 13 20:19:51.309804 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.309714 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hrhss"] Jul 13 20:19:51.309844 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.309806 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.313390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.311889 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-wmtv8\"" Jul 13 20:19:51.313390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.312519 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 13 20:19:51.313390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.312727 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 13 20:19:51.313390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.312888 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 13 20:19:51.367242 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367247 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367266 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367368 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367396 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367416 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzl24\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.367524 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.367498 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468318 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468223 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468318 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468267 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468412 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzl24\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468451 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/911cd773-52de-4103-9df0-21fd8acbc674-tmp-dir\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468499 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzmsw\" (UniqueName: \"kubernetes.io/projected/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-kube-api-access-jzmsw\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.468546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468531 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468608 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/911cd773-52de-4103-9df0-21fd8acbc674-config-volume\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-br5pf\" (UniqueName: \"kubernetes.io/projected/911cd773-52de-4103-9df0-21fd8acbc674-kube-api-access-br5pf\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.468907 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.468848 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-cert\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.469223 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.469001 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:19:51.469223 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.469017 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:19:51.469223 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.469034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.469223 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.469087 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:51.969069778 +0000 UTC m=+38.133629123 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:19:51.469438 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.469369 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.472735 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.472619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.472834 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.472670 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.477261 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.477239 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.477417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.477401 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzl24\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.478873 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.478854 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.569374 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569334 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/911cd773-52de-4103-9df0-21fd8acbc674-tmp-dir\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.569374 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569376 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jzmsw\" (UniqueName: \"kubernetes.io/projected/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-kube-api-access-jzmsw\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.569584 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569401 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.569584 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569431 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/911cd773-52de-4103-9df0-21fd8acbc674-config-volume\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.569584 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-br5pf\" (UniqueName: \"kubernetes.io/projected/911cd773-52de-4103-9df0-21fd8acbc674-kube-api-access-br5pf\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.569584 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569503 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-cert\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.569584 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.569508 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:19:51.569795 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.569598 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:52.069576578 +0000 UTC m=+38.234135915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:19:51.569795 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/911cd773-52de-4103-9df0-21fd8acbc674-tmp-dir\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.569988 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.569969 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/911cd773-52de-4103-9df0-21fd8acbc674-config-volume\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.572058 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.572039 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-cert\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.587195 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.587162 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzmsw\" (UniqueName: \"kubernetes.io/projected/d83a99b0-2e4c-4d72-bea4-97d78b414c7e-kube-api-access-jzmsw\") pod \"ingress-canary-4c2pr\" (UID: \"d83a99b0-2e4c-4d72-bea4-97d78b414c7e\") " pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.589046 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.589023 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-br5pf\" (UniqueName: \"kubernetes.io/projected/911cd773-52de-4103-9df0-21fd8acbc674-kube-api-access-br5pf\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:51.602720 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.602696 2580 generic.go:356] "Generic (PLEG): container finished" podID="81d105e4-a5b9-426c-988d-3988cf6b3706" containerID="a7f4b2bbcb3557aad881713f0f3c7b963a259436eb6f7548c9574d3f1d49312c" exitCode=0 Jul 13 20:19:51.602849 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.602729 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerDied","Data":"a7f4b2bbcb3557aad881713f0f3c7b963a259436eb6f7548c9574d3f1d49312c"} Jul 13 20:19:51.620180 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.620158 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-4c2pr" Jul 13 20:19:51.837515 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.837483 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-4c2pr"] Jul 13 20:19:51.842321 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:19:51.842296 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd83a99b0_2e4c_4d72_bea4_97d78b414c7e.slice/crio-3acbe4ad59b10a081b5c5f2d9787ef075d47c7fe961242665db8378384d2a880 WatchSource:0}: Error finding container 3acbe4ad59b10a081b5c5f2d9787ef075d47c7fe961242665db8378384d2a880: Status 404 returned error can't find the container with id 3acbe4ad59b10a081b5c5f2d9787ef075d47c7fe961242665db8378384d2a880 Jul 13 20:19:51.976207 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:51.976169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:51.976365 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.976313 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:19:51.976365 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.976328 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:19:51.976434 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:51.976376 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:52.976363833 +0000 UTC m=+39.140923162 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:19:52.076625 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.076547 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:52.076780 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:52.076712 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:19:52.076816 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:52.076794 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:53.076774225 +0000 UTC m=+39.241333566 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:19:52.334538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.334459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:19:52.334538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.334486 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:19:52.334766 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.334707 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:19:52.337238 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337188 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:19:52.337238 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337201 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 20:19:52.337238 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337230 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:19:52.337457 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337257 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j6z4f\"" Jul 13 20:19:52.337457 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337201 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-899ms\"" Jul 13 20:19:52.337544 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.337486 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 20:19:52.607387 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.607170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4c2pr" event={"ID":"d83a99b0-2e4c-4d72-bea4-97d78b414c7e","Type":"ContainerStarted","Data":"3acbe4ad59b10a081b5c5f2d9787ef075d47c7fe961242665db8378384d2a880"} Jul 13 20:19:52.611322 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.611298 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-tw92n" event={"ID":"81d105e4-a5b9-426c-988d-3988cf6b3706","Type":"ContainerStarted","Data":"e4ceb617ede6beb1de272adf0635df84ba849d8d0303e3c04c91f2e2e80bb6c6"} Jul 13 20:19:52.636200 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.636137 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-tw92n" podStartSLOduration=7.708056559 podStartE2EDuration="37.636117547s" totalImagesPullingTime="29.928060988s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:52.635392815 +0000 UTC m=+38.799952167" watchObservedRunningTime="2026-07-13 20:19:52.636117547 +0000 UTC m=+38.800676899" Jul 13 20:19:52.983182 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:52.983094 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:52.983393 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:52.983224 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:19:52.983393 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:52.983238 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:19:52.983393 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:52.983289 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:54.983275645 +0000 UTC m=+41.147834974 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:19:53.083930 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:53.083881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:53.084109 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:53.084023 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:19:53.084109 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:53.084083 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:55.084066058 +0000 UTC m=+41.248625389 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:19:54.618641 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:54.618607 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-4c2pr" event={"ID":"d83a99b0-2e4c-4d72-bea4-97d78b414c7e","Type":"ContainerStarted","Data":"e2b4d4fa12c418d677102e4353120b96030c106bba16fe303cc6fdcba3282d63"} Jul 13 20:19:54.634803 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:54.634713 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-4c2pr" podStartSLOduration=1.122705233 podStartE2EDuration="3.634696717s" totalImagesPullingTime="2.511991484s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:19:54.634567957 +0000 UTC m=+40.799127307" watchObservedRunningTime="2026-07-13 20:19:54.634696717 +0000 UTC m=+40.799256065" Jul 13 20:19:54.997875 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:54.997831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:54.998051 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:54.997985 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:19:54.998051 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:54.998006 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:19:54.998143 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:54.998060 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:58.998046411 +0000 UTC m=+45.162605743 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:19:55.098518 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:55.098484 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:55.098687 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:55.098640 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:19:55.098732 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:55.098705 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:59.098689828 +0000 UTC m=+45.263249160 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:19:58.861410 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.861375 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-7p5kz"] Jul 13 20:19:58.897659 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.897631 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7p5kz"] Jul 13 20:19:58.897819 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.897781 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:58.900294 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.900268 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 13 20:19:58.900497 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.900483 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 13 20:19:58.900882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.900867 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nlmqs\"" Jul 13 20:19:58.900929 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.900905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 13 20:19:58.901458 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.901441 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 13 20:19:58.928134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.928107 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:58.928225 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.928140 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcvrd\" (UniqueName: \"kubernetes.io/projected/072e919e-d0f4-4058-a026-414ec546d911-kube-api-access-jcvrd\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:58.928225 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.928167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/072e919e-d0f4-4058-a026-414ec546d911-data-volume\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:58.928225 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.928205 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:58.928327 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:58.928249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/072e919e-d0f4-4058-a026-414ec546d911-crio-socket\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.028840 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.028803 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.028936 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.028853 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:19:59.028936 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.028882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/072e919e-d0f4-4058-a026-414ec546d911-crio-socket\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.029030 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.028939 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.029030 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.028976 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:19:59.029030 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029013 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:19:59.029030 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029019 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:19:59.029030 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029027 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.029028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jcvrd\" (UniqueName: \"kubernetes.io/projected/072e919e-d0f4-4058-a026-414ec546d911-kube-api-access-jcvrd\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029063 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:59.529043224 +0000 UTC m=+45.693602555 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029080 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:07.029070929 +0000 UTC m=+53.193630261 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.029096 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:19:59.529089841 +0000 UTC m=+45.693649170 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : configmap "kube-rbac-proxy" not found Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.029112 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/072e919e-d0f4-4058-a026-414ec546d911-crio-socket\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.029242 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.029127 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/072e919e-d0f4-4058-a026-414ec546d911-data-volume\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.029448 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.029420 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/072e919e-d0f4-4058-a026-414ec546d911-data-volume\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.039233 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.039210 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcvrd\" (UniqueName: \"kubernetes.io/projected/072e919e-d0f4-4058-a026-414ec546d911-kube-api-access-jcvrd\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.129760 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.129656 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:19:59.129901 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.129826 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:19:59.129901 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.129889 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:07.129874865 +0000 UTC m=+53.294434195 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:19:59.532721 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.532631 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.532721 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:19:59.532686 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:19:59.532924 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.532777 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:19:59.532924 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.532801 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:19:59.532924 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.532844 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:00.532830772 +0000 UTC m=+46.697390101 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:19:59.532924 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:19:59.532857 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:00.532850355 +0000 UTC m=+46.697409683 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : configmap "kube-rbac-proxy" not found Jul 13 20:20:00.540626 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:00.540588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:00.541090 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:00.540716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:00.541090 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:00.540766 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:20:00.541090 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:00.540830 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:20:00.541090 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:00.540848 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:02.540830945 +0000 UTC m=+48.705390274 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:20:00.541090 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:00.540889 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:02.540876575 +0000 UTC m=+48.705435905 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : configmap "kube-rbac-proxy" not found Jul 13 20:20:01.413980 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:01.413955 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-69s4w_9853f083-cbfb-439a-8698-4f85d7007f17/dns-node-resolver/0.log" Jul 13 20:20:02.553963 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:02.553922 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:02.554399 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:02.554030 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:02.554399 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:02.554066 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:20:02.554399 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:02.554130 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:06.554115741 +0000 UTC m=+52.718675069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:20:02.554399 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:02.554134 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:20:02.554399 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:02.554184 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:06.554168624 +0000 UTC m=+52.718727963 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : configmap "kube-rbac-proxy" not found Jul 13 20:20:02.624460 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:02.624438 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9wtbh_c1146270-62c3-4f1e-b917-b3c45c509559/node-ca/0.log" Jul 13 20:20:03.213854 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:03.213826 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4c2pr_d83a99b0-2e4c-4d72-bea4-97d78b414c7e/serve-healthcheck-canary/0.log" Jul 13 20:20:06.584460 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:06.584419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:06.584933 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:06.584482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:06.584933 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:06.584545 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 13 20:20:06.584933 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:06.584596 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:20:06.584933 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:06.584609 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:14.584594126 +0000 UTC m=+60.749153459 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : configmap "kube-rbac-proxy" not found Jul 13 20:20:06.584933 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:06.584654 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:14.584640336 +0000 UTC m=+60.749199665 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:20:07.088592 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:07.088560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:07.088767 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:07.088713 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 13 20:20:07.088767 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:07.088732 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-7fd4cb9555-wlrqt: secret "image-registry-tls" not found Jul 13 20:20:07.088840 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:07.088807 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls podName:4f096483-33f1-4af3-984e-30775d8312c4 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:23.08878964 +0000 UTC m=+69.253348968 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls") pod "image-registry-7fd4cb9555-wlrqt" (UID: "4f096483-33f1-4af3-984e-30775d8312c4") : secret "image-registry-tls" not found Jul 13 20:20:07.189962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:07.189927 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:07.190104 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:07.190045 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 13 20:20:07.190104 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:07.190099 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls podName:911cd773-52de-4103-9df0-21fd8acbc674 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:23.190083248 +0000 UTC m=+69.354642589 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls") pod "dns-default-hrhss" (UID: "911cd773-52de-4103-9df0-21fd8acbc674") : secret "dns-default-metrics-tls" not found Jul 13 20:20:14.647053 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:14.647006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:14.647546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:14.647089 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:14.647546 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:14.647234 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 13 20:20:14.647546 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:14.647314 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls podName:072e919e-d0f4-4058-a026-414ec546d911 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:30.647298411 +0000 UTC m=+76.811857740 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls") pod "insights-runtime-extractor-7p5kz" (UID: "072e919e-d0f4-4058-a026-414ec546d911") : secret "insights-runtime-extractor-tls" not found Jul 13 20:20:14.647546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:14.647498 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/072e919e-d0f4-4058-a026-414ec546d911-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:16.595915 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:16.595889 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-4sksc" Jul 13 20:20:19.988214 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:19.988174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:20:19.991038 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:19.991016 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 13 20:20:20.001228 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.001204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0bd6ef49-c50c-44bc-b547-95958e3de5bb-original-pull-secret\") pod \"global-pull-secret-syncer-p2ps2\" (UID: \"0bd6ef49-c50c-44bc-b547-95958e3de5bb\") " pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:20:20.089379 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.089349 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:20:20.092055 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.092036 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 13 20:20:20.102124 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.102095 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/58ec379f-96e0-41d3-a09e-bdc06bd97c08-metrics-certs\") pod \"network-metrics-daemon-shvr5\" (UID: \"58ec379f-96e0-41d3-a09e-bdc06bd97c08\") " pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:20:20.190705 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.190665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:20:20.194026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.194007 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 13 20:20:20.204146 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.204125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 13 20:20:20.214806 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.214784 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4b4mx\" (UniqueName: \"kubernetes.io/projected/58065293-dd9f-4398-8e33-f48831027fda-kube-api-access-4b4mx\") pod \"network-check-target-ds8ss\" (UID: \"58065293-dd9f-4398-8e33-f48831027fda\") " pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:20:20.246899 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.246841 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-899ms\"" Jul 13 20:20:20.252241 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.252226 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j6z4f\"" Jul 13 20:20:20.255006 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.254986 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:20:20.255090 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.255005 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-p2ps2" Jul 13 20:20:20.260093 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.260077 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-shvr5" Jul 13 20:20:20.441442 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.441413 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-ds8ss"] Jul 13 20:20:20.445845 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:20.445817 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58065293_dd9f_4398_8e33_f48831027fda.slice/crio-032bf031ab4e1475af3170620382ebd156ee3dab38e6740e84edd892d892cbf7 WatchSource:0}: Error finding container 032bf031ab4e1475af3170620382ebd156ee3dab38e6740e84edd892d892cbf7: Status 404 returned error can't find the container with id 032bf031ab4e1475af3170620382ebd156ee3dab38e6740e84edd892d892cbf7 Jul 13 20:20:20.457857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.457829 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-p2ps2"] Jul 13 20:20:20.460165 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:20.460140 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0bd6ef49_c50c_44bc_b547_95958e3de5bb.slice/crio-bfda16e0f72f60d633b4dd2bc1eaf7612e91a25db677c0f9284b47ec5fa286d9 WatchSource:0}: Error finding container bfda16e0f72f60d633b4dd2bc1eaf7612e91a25db677c0f9284b47ec5fa286d9: Status 404 returned error can't find the container with id bfda16e0f72f60d633b4dd2bc1eaf7612e91a25db677c0f9284b47ec5fa286d9 Jul 13 20:20:20.481259 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.481226 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-shvr5"] Jul 13 20:20:20.485401 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:20.485375 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod58ec379f_96e0_41d3_a09e_bdc06bd97c08.slice/crio-ad5382515a51224f9c2b99684d2c101443a3c7dd2b96474a512ebf90d7ae7459 WatchSource:0}: Error finding container ad5382515a51224f9c2b99684d2c101443a3c7dd2b96474a512ebf90d7ae7459: Status 404 returned error can't find the container with id ad5382515a51224f9c2b99684d2c101443a3c7dd2b96474a512ebf90d7ae7459 Jul 13 20:20:20.681538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.681494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-shvr5" event={"ID":"58ec379f-96e0-41d3-a09e-bdc06bd97c08","Type":"ContainerStarted","Data":"ad5382515a51224f9c2b99684d2c101443a3c7dd2b96474a512ebf90d7ae7459"} Jul 13 20:20:20.682623 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.682596 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ds8ss" event={"ID":"58065293-dd9f-4398-8e33-f48831027fda","Type":"ContainerStarted","Data":"032bf031ab4e1475af3170620382ebd156ee3dab38e6740e84edd892d892cbf7"} Jul 13 20:20:20.683651 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:20.683628 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p2ps2" event={"ID":"0bd6ef49-c50c-44bc-b547-95958e3de5bb","Type":"ContainerStarted","Data":"bfda16e0f72f60d633b4dd2bc1eaf7612e91a25db677c0f9284b47ec5fa286d9"} Jul 13 20:20:22.691187 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.691150 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-shvr5" event={"ID":"58ec379f-96e0-41d3-a09e-bdc06bd97c08","Type":"ContainerStarted","Data":"b280dda6f3fb041140f2efa7dab11b9f5de03625739590c6dc8bab43589d30a9"} Jul 13 20:20:22.691641 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.691206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-shvr5" event={"ID":"58ec379f-96e0-41d3-a09e-bdc06bd97c08","Type":"ContainerStarted","Data":"8b57311fc26c848ac341868b15c656418f98e621b86c67a1be1ebfdc27648fba"} Jul 13 20:20:22.697138 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.697100 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fd4cb9555-wlrqt"] Jul 13 20:20:22.697346 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:22.697317 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" podUID="4f096483-33f1-4af3-984e-30775d8312c4" Jul 13 20:20:22.722701 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.722639 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-shvr5" podStartSLOduration=66.377095465 podStartE2EDuration="1m7.722615838s" totalImagesPullingTime="1.345520373s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:22.722398805 +0000 UTC m=+68.886958157" watchObservedRunningTime="2026-07-13 20:20:22.722615838 +0000 UTC m=+68.887175192" Jul 13 20:20:22.759619 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.759575 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm"] Jul 13 20:20:22.763719 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.763693 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-2jvd6"] Jul 13 20:20:22.763894 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.763835 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.766629 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.766605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:22.770159 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.770131 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 13 20:20:22.770843 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.770433 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 13 20:20:22.771142 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.771113 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 13 20:20:22.773850 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.773832 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 13 20:20:22.774185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.774161 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 13 20:20:22.774185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.774176 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 13 20:20:22.779795 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.779593 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-954hd\"" Jul 13 20:20:22.797076 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.797050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2jvd6"] Jul 13 20:20:22.798069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.798042 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm"] Jul 13 20:20:22.812312 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.812269 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1ba5eee-87d7-4988-a904-b62eb3b44a67-tmp\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.812453 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.812340 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1ba5eee-87d7-4988-a904-b62eb3b44a67-klusterlet-config\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.812453 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.812388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k28zt\" (UniqueName: \"kubernetes.io/projected/4211d7ed-40ee-437f-be9a-62d9a13694fb-kube-api-access-k28zt\") pod \"downloads-68c9cbb497-2jvd6\" (UID: \"4211d7ed-40ee-437f-be9a-62d9a13694fb\") " pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:22.812550 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.812469 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/a1ba5eee-87d7-4988-a904-b62eb3b44a67-kube-api-access-kqx8l\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.874110 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.874072 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-84fb4758cb-6fdhb"] Jul 13 20:20:22.878154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.878127 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.903572 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.903513 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84fb4758cb-6fdhb"] Jul 13 20:20:22.913838 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k28zt\" (UniqueName: \"kubernetes.io/projected/4211d7ed-40ee-437f-be9a-62d9a13694fb-kube-api-access-k28zt\") pod \"downloads-68c9cbb497-2jvd6\" (UID: \"4211d7ed-40ee-437f-be9a-62d9a13694fb\") " pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:22.913962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6f94186-5457-40c0-abec-3331eb1ba492-ca-trust-extracted\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.913962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-registry-certificates\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.913962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913934 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1ba5eee-87d7-4988-a904-b62eb3b44a67-klusterlet-config\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.914063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-registry-tls\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.913988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-bound-sa-token\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8trpc\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-kube-api-access-8trpc\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914163 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914061 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-trusted-ca\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914163 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/a1ba5eee-87d7-4988-a904-b62eb3b44a67-kube-api-access-kqx8l\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.914163 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-image-registry-private-configuration\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914163 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-installation-pull-secrets\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:22.914283 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1ba5eee-87d7-4988-a904-b62eb3b44a67-tmp\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.914627 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.914595 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a1ba5eee-87d7-4988-a904-b62eb3b44a67-tmp\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.917678 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.917651 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/a1ba5eee-87d7-4988-a904-b62eb3b44a67-klusterlet-config\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:22.930881 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.930858 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k28zt\" (UniqueName: \"kubernetes.io/projected/4211d7ed-40ee-437f-be9a-62d9a13694fb-kube-api-access-k28zt\") pod \"downloads-68c9cbb497-2jvd6\" (UID: \"4211d7ed-40ee-437f-be9a-62d9a13694fb\") " pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:22.931084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:22.931057 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kqx8l\" (UniqueName: \"kubernetes.io/projected/a1ba5eee-87d7-4988-a904-b62eb3b44a67-kube-api-access-kqx8l\") pod \"klusterlet-addon-workmgr-864db9d6d9-gcpwm\" (UID: \"a1ba5eee-87d7-4988-a904-b62eb3b44a67\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:23.015531 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015446 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6f94186-5457-40c0-abec-3331eb1ba492-ca-trust-extracted\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015531 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-registry-certificates\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015533 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-registry-tls\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015559 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-bound-sa-token\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015582 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8trpc\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-kube-api-access-8trpc\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015629 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-trusted-ca\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-image-registry-private-configuration\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015727 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015700 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-installation-pull-secrets\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.015996 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.015925 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/d6f94186-5457-40c0-abec-3331eb1ba492-ca-trust-extracted\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.017604 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.017549 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-trusted-ca\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.018312 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.018269 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/d6f94186-5457-40c0-abec-3331eb1ba492-registry-certificates\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.019155 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.019134 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-registry-tls\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.019299 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.019262 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-installation-pull-secrets\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.020632 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.020594 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/d6f94186-5457-40c0-abec-3331eb1ba492-image-registry-private-configuration\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.029993 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.029956 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-bound-sa-token\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.040063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.040038 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8trpc\" (UniqueName: \"kubernetes.io/projected/d6f94186-5457-40c0-abec-3331eb1ba492-kube-api-access-8trpc\") pod \"image-registry-84fb4758cb-6fdhb\" (UID: \"d6f94186-5457-40c0-abec-3331eb1ba492\") " pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.077483 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.077455 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:23.084734 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.084706 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:23.117077 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.117029 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:23.120087 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.120061 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"image-registry-7fd4cb9555-wlrqt\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:23.195439 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.195399 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-82c5x\"" Jul 13 20:20:23.199845 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.199808 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:23.217703 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.217664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:23.220327 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.220306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/911cd773-52de-4103-9df0-21fd8acbc674-metrics-tls\") pod \"dns-default-hrhss\" (UID: \"911cd773-52de-4103-9df0-21fd8acbc674\") " pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:23.396928 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.396894 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-t5qs6\"" Jul 13 20:20:23.404698 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.404664 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:23.694419 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.694339 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:23.702249 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.702223 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:23.824003 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.823966 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824029 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824063 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fzl24\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824086 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824121 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824370 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824228 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824370 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824270 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824370 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824299 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates\") pod \"4f096483-33f1-4af3-984e-30775d8312c4\" (UID: \"4f096483-33f1-4af3-984e-30775d8312c4\") " Jul 13 20:20:23.824513 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824451 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:20:23.824513 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824481 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:20:23.824622 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824596 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/4f096483-33f1-4af3-984e-30775d8312c4-ca-trust-extracted\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.824622 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824615 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-trusted-ca\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.824828 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.824808 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:20:23.827230 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.827181 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:20:23.827230 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.827194 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:20:23.827375 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.827270 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:20:23.827375 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.827299 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:20:23.827375 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.827353 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24" (OuterVolumeSpecName: "kube-api-access-fzl24") pod "4f096483-33f1-4af3-984e-30775d8312c4" (UID: "4f096483-33f1-4af3-984e-30775d8312c4"). InnerVolumeSpecName "kube-api-access-fzl24". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:20:23.925408 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925370 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-installation-pull-secrets\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.925408 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925402 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fzl24\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-kube-api-access-fzl24\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.925587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925415 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-bound-sa-token\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.925587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925429 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/4f096483-33f1-4af3-984e-30775d8312c4-registry-tls\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.925587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925441 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/4f096483-33f1-4af3-984e-30775d8312c4-image-registry-private-configuration\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:23.925587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:23.925452 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/4f096483-33f1-4af3-984e-30775d8312c4-registry-certificates\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:20:24.607233 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.607162 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:20:24.611571 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.611548 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.616641 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.616619 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 13 20:20:24.617279 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.617259 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 13 20:20:24.617390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.617373 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 13 20:20:24.618010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.617985 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-h8fpr\"" Jul 13 20:20:24.618225 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.618206 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 13 20:20:24.618419 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.618400 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 13 20:20:24.629820 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.629794 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:20:24.696593 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.696561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-7fd4cb9555-wlrqt" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733280 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733347 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733421 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgkcv\" (UniqueName: \"kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.735493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.733811 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-7fd4cb9555-wlrqt"] Jul 13 20:20:24.738183 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.738151 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-7fd4cb9555-wlrqt"] Jul 13 20:20:24.834617 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.834617 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834623 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.834864 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.834864 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.834864 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.834864 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.834792 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgkcv\" (UniqueName: \"kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.835769 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.835395 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.835769 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.835479 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.836000 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.835817 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.837508 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.837483 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.837715 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.837695 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.861399 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.861331 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgkcv\" (UniqueName: \"kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv\") pod \"console-85d8fb575f-qm8bp\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:24.922236 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:24.922195 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:25.416527 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.416498 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:20:25.417401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.417377 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-hrhss"] Jul 13 20:20:25.420684 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:25.420657 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24086d26_855b_4c41_baac_5c197dfef948.slice/crio-9c4a9c48025fbe8285186cf238af09fb640498f4c9d02fbb954a52d663444077 WatchSource:0}: Error finding container 9c4a9c48025fbe8285186cf238af09fb640498f4c9d02fbb954a52d663444077: Status 404 returned error can't find the container with id 9c4a9c48025fbe8285186cf238af09fb640498f4c9d02fbb954a52d663444077 Jul 13 20:20:25.421297 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:25.421276 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod911cd773_52de_4103_9df0_21fd8acbc674.slice/crio-c911fd5eef72c92b1dba89e001f7d710352228ce7fe44cd82681736d70556377 WatchSource:0}: Error finding container c911fd5eef72c92b1dba89e001f7d710352228ce7fe44cd82681736d70556377: Status 404 returned error can't find the container with id c911fd5eef72c92b1dba89e001f7d710352228ce7fe44cd82681736d70556377 Jul 13 20:20:25.634523 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.634367 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-84fb4758cb-6fdhb"] Jul 13 20:20:25.636925 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:25.636891 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd6f94186_5457_40c0_abec_3331eb1ba492.slice/crio-68d1044567ff84cd2d1d50b9b2c8ae67123cde422e282815f538af1df96b1400 WatchSource:0}: Error finding container 68d1044567ff84cd2d1d50b9b2c8ae67123cde422e282815f538af1df96b1400: Status 404 returned error can't find the container with id 68d1044567ff84cd2d1d50b9b2c8ae67123cde422e282815f538af1df96b1400 Jul 13 20:20:25.638385 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.638359 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm"] Jul 13 20:20:25.639229 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.639208 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-2jvd6"] Jul 13 20:20:25.641662 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:25.641630 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda1ba5eee_87d7_4988_a904_b62eb3b44a67.slice/crio-1e405ffa9cdafaa7cc7f21fbb393cef57a340d561f6a118e23e01eb7deab2ad5 WatchSource:0}: Error finding container 1e405ffa9cdafaa7cc7f21fbb393cef57a340d561f6a118e23e01eb7deab2ad5: Status 404 returned error can't find the container with id 1e405ffa9cdafaa7cc7f21fbb393cef57a340d561f6a118e23e01eb7deab2ad5 Jul 13 20:20:25.642310 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:25.642278 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4211d7ed_40ee_437f_be9a_62d9a13694fb.slice/crio-33520ae1a7e24caea2b463167c3da2b953bffcd11baea5f060588a8b2cd5607f WatchSource:0}: Error finding container 33520ae1a7e24caea2b463167c3da2b953bffcd11baea5f060588a8b2cd5607f: Status 404 returned error can't find the container with id 33520ae1a7e24caea2b463167c3da2b953bffcd11baea5f060588a8b2cd5607f Jul 13 20:20:25.701466 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.701437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" event={"ID":"a1ba5eee-87d7-4988-a904-b62eb3b44a67","Type":"ContainerStarted","Data":"1e405ffa9cdafaa7cc7f21fbb393cef57a340d561f6a118e23e01eb7deab2ad5"} Jul 13 20:20:25.702938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.702916 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-ds8ss" event={"ID":"58065293-dd9f-4398-8e33-f48831027fda","Type":"ContainerStarted","Data":"aba4525b8a6aa9e0e20c6dcd145e78d83e7fca81d0a921cc43f32372e51ea5aa"} Jul 13 20:20:25.703074 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.703043 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:20:25.704304 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.704284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" event={"ID":"d6f94186-5457-40c0-abec-3331eb1ba492","Type":"ContainerStarted","Data":"68d1044567ff84cd2d1d50b9b2c8ae67123cde422e282815f538af1df96b1400"} Jul 13 20:20:25.704401 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.704327 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:25.705441 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.705423 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrhss" event={"ID":"911cd773-52de-4103-9df0-21fd8acbc674","Type":"ContainerStarted","Data":"c911fd5eef72c92b1dba89e001f7d710352228ce7fe44cd82681736d70556377"} Jul 13 20:20:25.706954 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.706936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-p2ps2" event={"ID":"0bd6ef49-c50c-44bc-b547-95958e3de5bb","Type":"ContainerStarted","Data":"e6b0e0ba25cb31c93f913208c09a8ce3ae5ee1f1230b3a3e57511b22c97e61e7"} Jul 13 20:20:25.708176 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.708156 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2jvd6" event={"ID":"4211d7ed-40ee-437f-be9a-62d9a13694fb","Type":"ContainerStarted","Data":"33520ae1a7e24caea2b463167c3da2b953bffcd11baea5f060588a8b2cd5607f"} Jul 13 20:20:25.709276 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.709260 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8fb575f-qm8bp" event={"ID":"24086d26-855b-4c41-baac-5c197dfef948","Type":"ContainerStarted","Data":"9c4a9c48025fbe8285186cf238af09fb640498f4c9d02fbb954a52d663444077"} Jul 13 20:20:25.721100 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.721040 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-ds8ss" podStartSLOduration=65.969548006 podStartE2EDuration="1m10.721023438s" totalImagesPullingTime="4.751475432s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:25.720467072 +0000 UTC m=+71.885026419" watchObservedRunningTime="2026-07-13 20:20:25.721023438 +0000 UTC m=+71.885582798" Jul 13 20:20:25.742552 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.742509 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" podStartSLOduration=3.742495847 podStartE2EDuration="3.742495847s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:25.742084626 +0000 UTC m=+71.906643980" watchObservedRunningTime="2026-07-13 20:20:25.742495847 +0000 UTC m=+71.907055197" Jul 13 20:20:25.761672 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.761635 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-p2ps2" podStartSLOduration=66.005107509 podStartE2EDuration="1m10.761582365s" totalImagesPullingTime="4.756474856s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:25.760773165 +0000 UTC m=+71.925332513" watchObservedRunningTime="2026-07-13 20:20:25.761582365 +0000 UTC m=+71.926141694" Jul 13 20:20:25.875373 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.875341 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz"] Jul 13 20:20:25.878511 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.878487 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:25.881134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.880894 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 13 20:20:25.881134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.880917 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-j6t24\"" Jul 13 20:20:25.887251 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.887172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz"] Jul 13 20:20:25.946118 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:25.946086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6jjtz\" (UID: \"28430f79-afd3-46ad-8693-90644bf594c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:26.046689 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.046564 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6jjtz\" (UID: \"28430f79-afd3-46ad-8693-90644bf594c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:26.046876 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:26.046733 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 13 20:20:26.046876 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:20:26.046834 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates podName:28430f79-afd3-46ad-8693-90644bf594c2 nodeName:}" failed. No retries permitted until 2026-07-13 20:20:26.546811485 +0000 UTC m=+72.711370839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-6jjtz" (UID: "28430f79-afd3-46ad-8693-90644bf594c2") : secret "prometheus-operator-admission-webhook-tls" not found Jul 13 20:20:26.340760 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.340501 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4f096483-33f1-4af3-984e-30775d8312c4" path="/var/lib/kubelet/pods/4f096483-33f1-4af3-984e-30775d8312c4/volumes" Jul 13 20:20:26.551659 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.551142 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6jjtz\" (UID: \"28430f79-afd3-46ad-8693-90644bf594c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:26.554860 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.554833 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/28430f79-afd3-46ad-8693-90644bf594c2-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-6jjtz\" (UID: \"28430f79-afd3-46ad-8693-90644bf594c2\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:26.719577 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.719072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" event={"ID":"d6f94186-5457-40c0-abec-3331eb1ba492","Type":"ContainerStarted","Data":"fc552a87be95ffa33468dc39393194108bc0ecc5086730f75c0b1e58bf335bcd"} Jul 13 20:20:26.790628 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.790271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:26.966110 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:26.966076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz"] Jul 13 20:20:27.811215 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:27.811173 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28430f79_afd3_46ad_8693_90644bf594c2.slice/crio-46d1efd892b62eb565498850b88a7bff2d24e6eced2f98ca907b8c7b8978873e WatchSource:0}: Error finding container 46d1efd892b62eb565498850b88a7bff2d24e6eced2f98ca907b8c7b8978873e: Status 404 returned error can't find the container with id 46d1efd892b62eb565498850b88a7bff2d24e6eced2f98ca907b8c7b8978873e Jul 13 20:20:28.731037 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:28.730998 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" event={"ID":"28430f79-afd3-46ad-8693-90644bf594c2","Type":"ContainerStarted","Data":"46d1efd892b62eb565498850b88a7bff2d24e6eced2f98ca907b8c7b8978873e"} Jul 13 20:20:30.691545 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:30.691504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:30.694680 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:30.694646 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/072e919e-d0f4-4058-a026-414ec546d911-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-7p5kz\" (UID: \"072e919e-d0f4-4058-a026-414ec546d911\") " pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:30.713312 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:30.713283 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nlmqs\"" Jul 13 20:20:30.718559 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:30.718532 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-7p5kz" Jul 13 20:20:31.936957 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:31.936698 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-7p5kz"] Jul 13 20:20:32.750454 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.750413 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" event={"ID":"a1ba5eee-87d7-4988-a904-b62eb3b44a67","Type":"ContainerStarted","Data":"33bd76accdda2e1d1b3dbb70867b61899d31ae8b6f36e1dd02d7c256fddcf257"} Jul 13 20:20:32.751506 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.751481 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:32.754545 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.754516 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrhss" event={"ID":"911cd773-52de-4103-9df0-21fd8acbc674","Type":"ContainerStarted","Data":"5aefc6a67f63a682ca0c7ab3e2f8ab978994fcdd8033284689dded2eaa81a9ea"} Jul 13 20:20:32.754655 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.754572 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-hrhss" event={"ID":"911cd773-52de-4103-9df0-21fd8acbc674","Type":"ContainerStarted","Data":"08020fdcd21533aeeb2c6016513c2dae4420c218c56fbf1854abf388b0f1aee6"} Jul 13 20:20:32.755096 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.755073 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:32.757488 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.757334 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" Jul 13 20:20:32.758760 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.758667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p5kz" event={"ID":"072e919e-d0f4-4058-a026-414ec546d911","Type":"ContainerStarted","Data":"d08c8ce9da6e83cd7677b6f18aa1dddd85f713dc7d50a7f868eb6906f8ce4e7f"} Jul 13 20:20:32.762548 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.762521 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" event={"ID":"28430f79-afd3-46ad-8693-90644bf594c2","Type":"ContainerStarted","Data":"503ffd61c8d62429d85922bb2e6c73ebc78c3099e8ad094fc2beebfb8bef1928"} Jul 13 20:20:32.763041 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.763021 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:32.765954 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.765688 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8fb575f-qm8bp" event={"ID":"24086d26-855b-4c41-baac-5c197dfef948","Type":"ContainerStarted","Data":"01e892860714c0c68b766b226e040fc452cbbc194ce689aecb3859c2237fa364"} Jul 13 20:20:32.768960 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.768903 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" Jul 13 20:20:32.779712 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.779646 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-864db9d6d9-gcpwm" podStartSLOduration=4.654800333 podStartE2EDuration="10.779611872s" totalImagesPullingTime="6.124811539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:32.777161164 +0000 UTC m=+78.941720515" watchObservedRunningTime="2026-07-13 20:20:32.779611872 +0000 UTC m=+78.944171225" Jul 13 20:20:32.839711 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.839652 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-6jjtz" podStartSLOduration=3.877992335 podStartE2EDuration="7.839631762s" totalImagesPullingTime="3.961639427s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:32.838900101 +0000 UTC m=+79.003459455" watchObservedRunningTime="2026-07-13 20:20:32.839631762 +0000 UTC m=+79.004191115" Jul 13 20:20:32.840839 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:32.840795 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-hrhss" podStartSLOduration=35.525683327 podStartE2EDuration="41.840777286s" totalImagesPullingTime="6.315093959s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:19:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:32.802716543 +0000 UTC m=+78.967275898" watchObservedRunningTime="2026-07-13 20:20:32.840777286 +0000 UTC m=+79.005336637" Jul 13 20:20:33.946892 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.946833 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-85d8fb575f-qm8bp" podStartSLOduration=3.602930492 podStartE2EDuration="9.946810708s" totalImagesPullingTime="6.343880216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:32.892831538 +0000 UTC m=+79.057390889" watchObservedRunningTime="2026-07-13 20:20:33.946810708 +0000 UTC m=+80.111370065" Jul 13 20:20:33.948488 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.947371 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qrk7d"] Jul 13 20:20:33.958417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.957510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:33.960592 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 13 20:20:33.960592 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-nr54z\"" Jul 13 20:20:33.960592 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960512 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 13 20:20:33.960876 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960642 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 13 20:20:33.960876 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960777 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 13 20:20:33.960876 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.960853 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 13 20:20:33.964734 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:33.964712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qrk7d"] Jul 13 20:20:34.017013 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.016981 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.017166 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.017047 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4def49c7-958f-4147-8a0d-9a5cb255c72b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.017166 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.017067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.017166 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.017092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjl6q\" (UniqueName: \"kubernetes.io/projected/4def49c7-958f-4147-8a0d-9a5cb255c72b-kube-api-access-mjl6q\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.117852 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.117805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4def49c7-958f-4147-8a0d-9a5cb255c72b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.117852 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.117855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.118072 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.117882 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mjl6q\" (UniqueName: \"kubernetes.io/projected/4def49c7-958f-4147-8a0d-9a5cb255c72b-kube-api-access-mjl6q\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.118072 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.117959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.119503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.119475 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/4def49c7-958f-4147-8a0d-9a5cb255c72b-metrics-client-ca\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.121042 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.120996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.121167 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.121071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/4def49c7-958f-4147-8a0d-9a5cb255c72b-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.126962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.126915 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjl6q\" (UniqueName: \"kubernetes.io/projected/4def49c7-958f-4147-8a0d-9a5cb255c72b-kube-api-access-mjl6q\") pod \"prometheus-operator-64785b6675-qrk7d\" (UID: \"4def49c7-958f-4147-8a0d-9a5cb255c72b\") " pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.270569 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.270499 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" Jul 13 20:20:34.922541 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.922506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:34.922720 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.922593 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:34.924301 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.924007 2580 patch_prober.go:28] interesting pod/console-85d8fb575f-qm8bp container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" start-of-body= Jul 13 20:20:34.924301 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:34.924057 2580 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-85d8fb575f-qm8bp" podUID="24086d26-855b-4c41-baac-5c197dfef948" containerName="console" probeResult="failure" output="Get \"https://10.133.0.13:8443/health\": dial tcp 10.133.0.13:8443: connect: connection refused" Jul 13 20:20:36.275610 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:36.275510 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-qrk7d"] Jul 13 20:20:36.281119 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:36.281089 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4def49c7_958f_4147_8a0d_9a5cb255c72b.slice/crio-8da01ffbb89fcf53c75b51645e9560ed55c0d779c93a76f8cc1ff4ba75ed3e41 WatchSource:0}: Error finding container 8da01ffbb89fcf53c75b51645e9560ed55c0d779c93a76f8cc1ff4ba75ed3e41: Status 404 returned error can't find the container with id 8da01ffbb89fcf53c75b51645e9560ed55c0d779c93a76f8cc1ff4ba75ed3e41 Jul 13 20:20:36.785514 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:36.785470 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" event={"ID":"4def49c7-958f-4147-8a0d-9a5cb255c72b","Type":"ContainerStarted","Data":"8da01ffbb89fcf53c75b51645e9560ed55c0d779c93a76f8cc1ff4ba75ed3e41"} Jul 13 20:20:36.789349 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:36.789315 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p5kz" event={"ID":"072e919e-d0f4-4058-a026-414ec546d911","Type":"ContainerStarted","Data":"febed182f0865801bd44e5dca3eab8e20a1666025492901da8fe3c27d132d494"} Jul 13 20:20:43.205769 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:43.205701 2580 patch_prober.go:28] interesting pod/image-registry-84fb4758cb-6fdhb container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 13 20:20:43.206250 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:43.205785 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" podUID="d6f94186-5457-40c0-abec-3331eb1ba492" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 13 20:20:43.776239 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:43.776200 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-hrhss" Jul 13 20:20:44.927300 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:44.927279 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:44.931673 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:44.931649 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:20:45.551391 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.551350 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:20:45.829293 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.829177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-2jvd6" event={"ID":"4211d7ed-40ee-437f-be9a-62d9a13694fb","Type":"ContainerStarted","Data":"43f34318f6d501d69fd01c4e3689ed7acfdbdc90d30e3b7862ddb3aa4d32ff7b"} Jul 13 20:20:45.829733 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.829468 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:45.832499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.832460 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" event={"ID":"4def49c7-958f-4147-8a0d-9a5cb255c72b","Type":"ContainerStarted","Data":"02e508cff0cc1b01c37f4c2ed313989e1e3b47e0ee3b090765c78b87575c10a8"} Jul 13 20:20:45.832654 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.832514 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" event={"ID":"4def49c7-958f-4147-8a0d-9a5cb255c72b","Type":"ContainerStarted","Data":"11fbe146beb427b6a6900aa7daa31da124e6ad3dfcc43c47a5ecec178e2a2368"} Jul 13 20:20:45.834886 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.834858 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p5kz" event={"ID":"072e919e-d0f4-4058-a026-414ec546d911","Type":"ContainerStarted","Data":"b3d63ab212679df4455a1bdf5272c38396c49c2eaf3f905790a9ebd2eaf0fe46"} Jul 13 20:20:45.843686 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.843662 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-2jvd6" Jul 13 20:20:45.865394 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.864812 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-2jvd6" podStartSLOduration=4.710491805 podStartE2EDuration="23.864793489s" totalImagesPullingTime="19.154301684s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:45.846930581 +0000 UTC m=+92.011489933" watchObservedRunningTime="2026-07-13 20:20:45.864793489 +0000 UTC m=+92.029352842" Jul 13 20:20:45.865394 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:45.864919 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-qrk7d" podStartSLOduration=4.348271019 podStartE2EDuration="12.864907501s" totalImagesPullingTime="8.516636482s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:45.863473643 +0000 UTC m=+92.028033011" watchObservedRunningTime="2026-07-13 20:20:45.864907501 +0000 UTC m=+92.029466852" Jul 13 20:20:47.666505 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.666469 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q"] Jul 13 20:20:47.736663 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.736628 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-84fb4758cb-6fdhb" Jul 13 20:20:47.736934 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.736676 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlfjp"] Jul 13 20:20:47.738930 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.738896 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.741386 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.740964 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7m487\"" Jul 13 20:20:47.741386 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.741332 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 13 20:20:47.741386 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.741375 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 13 20:20:47.764508 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.764464 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q"] Jul 13 20:20:47.764651 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.764541 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlfjp"] Jul 13 20:20:47.764651 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.764581 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-b4d5g"] Jul 13 20:20:47.764788 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.764685 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.768194 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.767363 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 13 20:20:47.768194 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.767686 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-vp47h\"" Jul 13 20:20:47.768194 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.767977 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 13 20:20:47.768899 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.768788 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 13 20:20:47.799145 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.799107 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.803498 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.803411 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 13 20:20:47.803695 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.803655 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-pks4v\"" Jul 13 20:20:47.803901 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.803886 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 13 20:20:47.804610 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.804424 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 13 20:20:47.836534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.836506 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.836829 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.836805 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mv8w6\" (UniqueName: \"kubernetes.io/projected/97efcf0a-cf78-411f-8548-2efc4d7e3563-kube-api-access-mv8w6\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.836919 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.836843 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97efcf0a-cf78-411f-8548-2efc4d7e3563-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.837050 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.837013 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.938332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938250 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-slzpl\" (UniqueName: \"kubernetes.io/projected/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-kube-api-access-slzpl\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938336 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-metrics-client-ca\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938366 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-sys\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/91168a65-5bd2-4517-82e0-9e3b7174cd31-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938460 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.938534 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mv8w6\" (UniqueName: \"kubernetes.io/projected/97efcf0a-cf78-411f-8548-2efc4d7e3563-kube-api-access-mv8w6\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938584 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938610 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hng8r\" (UniqueName: \"kubernetes.io/projected/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-api-access-hng8r\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97efcf0a-cf78-411f-8548-2efc4d7e3563-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938692 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-root\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938722 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-textfile\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938767 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-tls\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.938857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938852 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:47.939332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.938876 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-wtmp\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:47.940097 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.940032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97efcf0a-cf78-411f-8548-2efc4d7e3563-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.946519 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.946471 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.955570 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.955540 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mv8w6\" (UniqueName: \"kubernetes.io/projected/97efcf0a-cf78-411f-8548-2efc4d7e3563-kube-api-access-mv8w6\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:47.962599 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:47.962568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97efcf0a-cf78-411f-8548-2efc4d7e3563-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-9cd2q\" (UID: \"97efcf0a-cf78-411f-8548-2efc4d7e3563\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:48.039548 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hng8r\" (UniqueName: \"kubernetes.io/projected/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-api-access-hng8r\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-root\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-textfile\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039703 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-tls\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.039768 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039722 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039772 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-wtmp\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-slzpl\" (UniqueName: \"kubernetes.io/projected/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-kube-api-access-slzpl\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039867 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-metrics-client-ca\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039923 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-sys\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/91168a65-5bd2-4517-82e0-9e3b7174cd31-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.040135 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.039981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.041273 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041224 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-sys\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.041273 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041238 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-accelerators-collector-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.041772 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-metrics-client-ca\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.041909 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041819 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/91168a65-5bd2-4517-82e0-9e3b7174cd31-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.041990 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-wtmp\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.042133 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.041704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.042252 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.042229 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-root\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.042476 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.042456 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-textfile\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.042553 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.042525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/91168a65-5bd2-4517-82e0-9e3b7174cd31-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.043848 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.043566 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.045036 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.044946 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-node-exporter-tls\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.045296 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.045271 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.046291 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.046267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.052207 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.052021 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" Jul 13 20:20:48.052395 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.052372 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-slzpl\" (UniqueName: \"kubernetes.io/projected/e6bb5201-1df2-4b42-84ce-25749bfc8e8f-kube-api-access-slzpl\") pod \"node-exporter-b4d5g\" (UID: \"e6bb5201-1df2-4b42-84ce-25749bfc8e8f\") " pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.061629 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.054173 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hng8r\" (UniqueName: \"kubernetes.io/projected/91168a65-5bd2-4517-82e0-9e3b7174cd31-kube-api-access-hng8r\") pod \"kube-state-metrics-785778ff74-dlfjp\" (UID: \"91168a65-5bd2-4517-82e0-9e3b7174cd31\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.079885 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.079861 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" Jul 13 20:20:48.113876 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.113853 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-b4d5g" Jul 13 20:20:48.245984 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:48.245950 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6bb5201_1df2_4b42_84ce_25749bfc8e8f.slice/crio-600279c684c765d2357b509ea9e28224773678f973feaf4c5abc1aaf6f3ad497 WatchSource:0}: Error finding container 600279c684c765d2357b509ea9e28224773678f973feaf4c5abc1aaf6f3ad497: Status 404 returned error can't find the container with id 600279c684c765d2357b509ea9e28224773678f973feaf4c5abc1aaf6f3ad497 Jul 13 20:20:48.424485 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.424452 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-dlfjp"] Jul 13 20:20:48.428481 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:48.428451 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91168a65_5bd2_4517_82e0_9e3b7174cd31.slice/crio-18fc64cb4944f6636b994fe5d83764e7c97d4354b628d84ebc48efead27d7968 WatchSource:0}: Error finding container 18fc64cb4944f6636b994fe5d83764e7c97d4354b628d84ebc48efead27d7968: Status 404 returned error can't find the container with id 18fc64cb4944f6636b994fe5d83764e7c97d4354b628d84ebc48efead27d7968 Jul 13 20:20:48.441665 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.441611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q"] Jul 13 20:20:48.445199 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:48.445175 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97efcf0a_cf78_411f_8548_2efc4d7e3563.slice/crio-c78ce584c9620c7d7601234d388e19bbce5548562601744e1f706f5547ebca49 WatchSource:0}: Error finding container c78ce584c9620c7d7601234d388e19bbce5548562601744e1f706f5547ebca49: Status 404 returned error can't find the container with id c78ce584c9620c7d7601234d388e19bbce5548562601744e1f706f5547ebca49 Jul 13 20:20:48.450578 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.450556 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:20:48.477928 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.477903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:20:48.478063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.478029 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.481081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.480863 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 13 20:20:48.481081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.480878 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 13 20:20:48.481081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.480877 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 13 20:20:48.481081 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481048 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 13 20:20:48.481344 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481117 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 13 20:20:48.481344 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481216 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-qqms6\"" Jul 13 20:20:48.481344 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481262 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 13 20:20:48.481811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481629 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 13 20:20:48.481811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.481765 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 13 20:20:48.482280 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.482126 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 13 20:20:48.646525 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646525 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-web-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646573 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646772 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-volume\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646966 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646966 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646966 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646922 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.646966 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646952 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8ccmw\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-kube-api-access-8ccmw\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.647143 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.646968 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.647143 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.647040 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.647143 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.647091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-out\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748300 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748246 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8ccmw\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-kube-api-access-8ccmw\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748385 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748421 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748453 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-out\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748487 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.748633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-web-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.749009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748677 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.749009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.749009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.749009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748890 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-volume\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.749009 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.748916 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.750603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.749508 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.750603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.749697 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/943e3fe8-5e85-44f4-b987-77f8f7f30929-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.750603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.750505 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.751834 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.751727 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.751983 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.751957 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-tls-assets\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.752373 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.752348 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.752705 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.752682 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.754278 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.753847 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-volume\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.754278 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.754232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.754549 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.754529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-web-config\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.763650 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.760097 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/943e3fe8-5e85-44f4-b987-77f8f7f30929-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.763650 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.763516 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-8ccmw\" (UniqueName: \"kubernetes.io/projected/943e3fe8-5e85-44f4-b987-77f8f7f30929-kube-api-access-8ccmw\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.766493 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.764873 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/943e3fe8-5e85-44f4-b987-77f8f7f30929-config-out\") pod \"alertmanager-main-0\" (UID: \"943e3fe8-5e85-44f4-b987-77f8f7f30929\") " pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.791827 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.791794 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 13 20:20:48.848785 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.848733 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-7p5kz" event={"ID":"072e919e-d0f4-4058-a026-414ec546d911","Type":"ContainerStarted","Data":"a423310663e4c64baf0533b5c92f326285bcc425879eb797f1553225760e81d5"} Jul 13 20:20:48.851517 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.851435 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4d5g" event={"ID":"e6bb5201-1df2-4b42-84ce-25749bfc8e8f","Type":"ContainerStarted","Data":"600279c684c765d2357b509ea9e28224773678f973feaf4c5abc1aaf6f3ad497"} Jul 13 20:20:48.854905 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.854802 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" event={"ID":"97efcf0a-cf78-411f-8548-2efc4d7e3563","Type":"ContainerStarted","Data":"c5c34dc92d6a5de19afba61993ec9d9db95a125c4a0d592167f89517bd58e69d"} Jul 13 20:20:48.854905 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.854856 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" event={"ID":"97efcf0a-cf78-411f-8548-2efc4d7e3563","Type":"ContainerStarted","Data":"c78ce584c9620c7d7601234d388e19bbce5548562601744e1f706f5547ebca49"} Jul 13 20:20:48.856352 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.856331 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" event={"ID":"91168a65-5bd2-4517-82e0-9e3b7174cd31","Type":"ContainerStarted","Data":"18fc64cb4944f6636b994fe5d83764e7c97d4354b628d84ebc48efead27d7968"} Jul 13 20:20:48.967715 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.967612 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-7p5kz" podStartSLOduration=34.379590606 podStartE2EDuration="50.967588639s" totalImagesPullingTime="16.587998033s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 20:19:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:48.873525049 +0000 UTC m=+95.038084403" watchObservedRunningTime="2026-07-13 20:20:48.967588639 +0000 UTC m=+95.132147992" Jul 13 20:20:48.967996 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:48.967973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 13 20:20:48.973779 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:48.973282 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod943e3fe8_5e85_44f4_b987_77f8f7f30929.slice/crio-e5138365273d16621ece04c60e08d15f494ad0e54318fc2c8ec6f99e34bafae8 WatchSource:0}: Error finding container e5138365273d16621ece04c60e08d15f494ad0e54318fc2c8ec6f99e34bafae8: Status 404 returned error can't find the container with id e5138365273d16621ece04c60e08d15f494ad0e54318fc2c8ec6f99e34bafae8 Jul 13 20:20:49.866109 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:49.866005 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" event={"ID":"97efcf0a-cf78-411f-8548-2efc4d7e3563","Type":"ContainerStarted","Data":"81f9a0672eb0171fcefc0f7a87e7069cf2d1069bcb9bdb2335ade97589ebe60d"} Jul 13 20:20:49.871180 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:49.871088 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"e5138365273d16621ece04c60e08d15f494ad0e54318fc2c8ec6f99e34bafae8"} Jul 13 20:20:50.429838 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.429802 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-7d76897594-gcznm"] Jul 13 20:20:50.449101 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.448861 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d76897594-gcznm"] Jul 13 20:20:50.449101 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.449016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.452423 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.452397 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 13 20:20:50.452809 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.452709 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 13 20:20:50.452809 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.452785 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-2tcgj3rvisr22\"" Jul 13 20:20:50.452972 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.452954 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 13 20:20:50.453477 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.453188 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-zpdjr\"" Jul 13 20:20:50.453477 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.453028 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 13 20:20:50.453477 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.453097 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 13 20:20:50.566126 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566089 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566289 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-grpc-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566289 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566208 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566428 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566300 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566428 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566428 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566402 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fzrps\" (UniqueName: \"kubernetes.io/projected/635d7e74-d757-47ef-acd3-ff527b8a17d9-kube-api-access-fzrps\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566557 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566437 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.566557 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.566535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635d7e74-d757-47ef-acd3-ff527b8a17d9-metrics-client-ca\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.667896 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.667858 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.667913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-grpc-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.667949 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.668005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.668038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.668061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fzrps\" (UniqueName: \"kubernetes.io/projected/635d7e74-d757-47ef-acd3-ff527b8a17d9-kube-api-access-fzrps\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668346 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.668098 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.668346 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.668148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635d7e74-d757-47ef-acd3-ff527b8a17d9-metrics-client-ca\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.669078 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.669026 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/635d7e74-d757-47ef-acd3-ff527b8a17d9-metrics-client-ca\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.673178 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.672895 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.673752 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.673708 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.673867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.673814 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-grpc-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.674277 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.674069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.674903 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.674857 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.678180 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.678155 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fzrps\" (UniqueName: \"kubernetes.io/projected/635d7e74-d757-47ef-acd3-ff527b8a17d9-kube-api-access-fzrps\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.683405 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.683351 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/635d7e74-d757-47ef-acd3-ff527b8a17d9-secret-thanos-querier-tls\") pod \"thanos-querier-7d76897594-gcznm\" (UID: \"635d7e74-d757-47ef-acd3-ff527b8a17d9\") " pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:50.762786 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:50.762758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:52.129497 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.128493 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:20:52.160406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.159426 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-7d76897594-gcznm"] Jul 13 20:20:52.160406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.159502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:20:52.160406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.159605 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.178096 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.177832 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290732 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xm598\" (UniqueName: \"kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290778 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290840 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.291044 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.290906 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392328 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392061 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392328 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392125 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392328 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392173 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392582 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392455 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xm598\" (UniqueName: \"kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392582 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392514 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392692 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392591 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392692 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392622 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.392965 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392938 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.393093 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.392948 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.393830 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.393800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.394058 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.394036 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.395898 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.395833 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.396723 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.396701 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.410341 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.410321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xm598\" (UniqueName: \"kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598\") pod \"console-554cbc8db6-szf9z\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.501526 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.501240 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:20:52.689857 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.689819 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:20:52.697872 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:20:52.697839 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod93f8f81b_4375_47b9_a9a0_e3ae524577c6.slice/crio-752f264a82b75edaaf0b6a7ab7e5d126ecbd1fee8335cb10b6f1eab7d7fbd930 WatchSource:0}: Error finding container 752f264a82b75edaaf0b6a7ab7e5d126ecbd1fee8335cb10b6f1eab7d7fbd930: Status 404 returned error can't find the container with id 752f264a82b75edaaf0b6a7ab7e5d126ecbd1fee8335cb10b6f1eab7d7fbd930 Jul 13 20:20:52.889138 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.888940 2580 generic.go:356] "Generic (PLEG): container finished" podID="943e3fe8-5e85-44f4-b987-77f8f7f30929" containerID="474afe13c4497b68052e00bcf868615975fb7fee1d9d81ec7998b2b2d3bd44a2" exitCode=0 Jul 13 20:20:52.889138 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.889049 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerDied","Data":"474afe13c4497b68052e00bcf868615975fb7fee1d9d81ec7998b2b2d3bd44a2"} Jul 13 20:20:52.895307 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.895266 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" event={"ID":"97efcf0a-cf78-411f-8548-2efc4d7e3563","Type":"ContainerStarted","Data":"1e0f7256409d616ba1a66c975797a6b8914ec750e2a8412256a497e7c80d1fcf"} Jul 13 20:20:52.898687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.898606 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"c42e376e492b91489533a3d5fe58d7a885040e6d059d5ad937861183aa7a76e3"} Jul 13 20:20:52.902097 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.902067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" event={"ID":"91168a65-5bd2-4517-82e0-9e3b7174cd31","Type":"ContainerStarted","Data":"534ba502eda34ef4e6012bd3e456b315414234d178f4915afcf3b31b431b7a43"} Jul 13 20:20:52.902196 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.902122 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" event={"ID":"91168a65-5bd2-4517-82e0-9e3b7174cd31","Type":"ContainerStarted","Data":"37a0e1be4f888845f770a692ef6343bfeca8e830e9b9021e8f654df8f52ee870"} Jul 13 20:20:52.902196 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.902153 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" event={"ID":"91168a65-5bd2-4517-82e0-9e3b7174cd31","Type":"ContainerStarted","Data":"436ce2ef8c04b71c3a5b9be5fd9542eca49d205882066c7a3e7b9a00895d182d"} Jul 13 20:20:52.904650 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.904625 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554cbc8db6-szf9z" event={"ID":"93f8f81b-4375-47b9-a9a0-e3ae524577c6","Type":"ContainerStarted","Data":"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16"} Jul 13 20:20:52.904784 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.904667 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554cbc8db6-szf9z" event={"ID":"93f8f81b-4375-47b9-a9a0-e3ae524577c6","Type":"ContainerStarted","Data":"752f264a82b75edaaf0b6a7ab7e5d126ecbd1fee8335cb10b6f1eab7d7fbd930"} Jul 13 20:20:52.906653 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.906326 2580 generic.go:356] "Generic (PLEG): container finished" podID="e6bb5201-1df2-4b42-84ce-25749bfc8e8f" containerID="c51abb559dad897ab561c7435d82616e4e4c0404c8839d24c840e2489fce9dd6" exitCode=0 Jul 13 20:20:52.906653 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.906358 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4d5g" event={"ID":"e6bb5201-1df2-4b42-84ce-25749bfc8e8f","Type":"ContainerDied","Data":"c51abb559dad897ab561c7435d82616e4e4c0404c8839d24c840e2489fce9dd6"} Jul 13 20:20:52.957791 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:52.957246 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-554cbc8db6-szf9z" podStartSLOduration=0.957223288 podStartE2EDuration="957.223288ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:52.955192302 +0000 UTC m=+99.119751657" watchObservedRunningTime="2026-07-13 20:20:52.957223288 +0000 UTC m=+99.121782645" Jul 13 20:20:53.009373 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:53.009266 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-dlfjp" podStartSLOduration=2.510340319 podStartE2EDuration="6.00924422s" totalImagesPullingTime="3.498903901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:52.986830674 +0000 UTC m=+99.151390025" watchObservedRunningTime="2026-07-13 20:20:53.00924422 +0000 UTC m=+99.173803566" Jul 13 20:20:53.029213 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:53.028308 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-9cd2q" podStartSLOduration=3.029734127 podStartE2EDuration="6.028287863s" totalImagesPullingTime="2.998553736s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:53.026511814 +0000 UTC m=+99.191071190" watchObservedRunningTime="2026-07-13 20:20:53.028287863 +0000 UTC m=+99.192847218" Jul 13 20:20:53.917263 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:53.917165 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4d5g" event={"ID":"e6bb5201-1df2-4b42-84ce-25749bfc8e8f","Type":"ContainerStarted","Data":"76cc236dbfee6c5a9efc55c85611547baefa55953db045458fd9873485081097"} Jul 13 20:20:53.917944 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:53.917276 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-b4d5g" event={"ID":"e6bb5201-1df2-4b42-84ce-25749bfc8e8f","Type":"ContainerStarted","Data":"d962d5a49a1be7413326ad7f3cd218a21ea5f74063fb9a2818b86862d31920fb"} Jul 13 20:20:53.939605 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:53.939501 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-b4d5g" podStartSLOduration=3.089173056 podStartE2EDuration="6.93948198s" totalImagesPullingTime="3.692951107s" totalInitContainerRuntime="157.357817ms" isStatefulPod=true podCreationTimestamp="2026-07-13 20:20:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:53.937810215 +0000 UTC m=+100.102369567" watchObservedRunningTime="2026-07-13 20:20:53.93948198 +0000 UTC m=+100.104041331" Jul 13 20:20:55.928065 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.928030 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"cbb1804ca84c4524f1604b5edae3d2ae189da1601ebeafb7a96dffa66b9c5200"} Jul 13 20:20:55.928487 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.928142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"4f389fb272ac54c0b2a721a4fc39bc199179cf00423c55ea5e909c693a162324"} Jul 13 20:20:55.928487 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.928182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"c511cdedd98da597f8f4d8e3df45346d8f9a7469f39875d69d6ec41fc9ea948d"} Jul 13 20:20:55.930603 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.930576 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"8763b16fb36f5b0e1189d2b562f18f3790cda2dfe2e8502edc281fb1e8ed4715"} Jul 13 20:20:55.930766 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.930652 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"cc1d499ef741d15fd6c4f3e4a5c4af93c0b52cbcd8e49a8996a95de80356d6cb"} Jul 13 20:20:55.930766 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:55.930685 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"dff88b90003e1b26feb3c6735c6232eef92b77cb5bcb6faca174f105e0aac70e"} Jul 13 20:20:56.722760 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:56.722705 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-ds8ss" Jul 13 20:20:56.941034 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:56.941001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"b497e29c400708dd55ceee906ecff5f87e7298016957e540876980141b2f63b3"} Jul 13 20:20:56.941460 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:56.941059 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"53b36b90b124d50a20115be32a7ec95921528d236fc391e9b779f595860ab94b"} Jul 13 20:20:58.952996 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.952964 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"943e3fe8-5e85-44f4-b987-77f8f7f30929","Type":"ContainerStarted","Data":"33a46f56ece73ddba47e303200b8db79cb807584a56671aa82b084af70d3eb24"} Jul 13 20:20:58.955853 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.955821 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"1559bc1bc1a0c34847addfcdb7ec36fe68b983b1fe060ee3e2d2c1fc1a6b7984"} Jul 13 20:20:58.955961 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.955875 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"40374cc83f5d1ac6711de9d4dbf6376e977db363d021d0b5ec771028bc51fd22"} Jul 13 20:20:58.955961 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.955904 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" event={"ID":"635d7e74-d757-47ef-acd3-ff527b8a17d9","Type":"ContainerStarted","Data":"a543409c82577cc430681d5956f35c7a2cd178a2ea81325fd70d9461d523826a"} Jul 13 20:20:58.955961 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.955938 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:20:58.982145 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:58.982098 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.706787779 podStartE2EDuration="10.982083729s" totalImagesPullingTime="7.245520339s" totalInitContainerRuntime="29.775611ms" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:48 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:58.979897716 +0000 UTC m=+105.144457066" watchObservedRunningTime="2026-07-13 20:20:58.982083729 +0000 UTC m=+105.146643080" Jul 13 20:20:59.004224 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:20:59.004173 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" podStartSLOduration=3.569799701 podStartE2EDuration="9.004153195s" totalImagesPullingTime="5.434353494s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:20:50 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:20:59.001836434 +0000 UTC m=+105.166395785" watchObservedRunningTime="2026-07-13 20:20:59.004153195 +0000 UTC m=+105.168712547" Jul 13 20:21:02.501625 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:02.501593 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:21:02.502223 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:02.501645 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:21:02.506482 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:02.506460 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:21:02.976171 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:02.976145 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:21:04.965483 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:04.965454 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-7d76897594-gcznm" Jul 13 20:21:11.867676 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:11.867620 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-85d8fb575f-qm8bp" podUID="24086d26-855b-4c41-baac-5c197dfef948" containerName="console" containerID="cri-o://01e892860714c0c68b766b226e040fc452cbbc194ce689aecb3859c2237fa364" gracePeriod=15 Jul 13 20:21:12.014290 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.014251 2580 generic.go:356] "Generic (PLEG): container finished" podID="24086d26-855b-4c41-baac-5c197dfef948" containerID="01e892860714c0c68b766b226e040fc452cbbc194ce689aecb3859c2237fa364" exitCode=0 Jul 13 20:21:12.014412 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.014321 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8fb575f-qm8bp" event={"ID":"24086d26-855b-4c41-baac-5c197dfef948","Type":"ContainerDied","Data":"01e892860714c0c68b766b226e040fc452cbbc194ce689aecb3859c2237fa364"} Jul 13 20:21:12.152275 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.152247 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:21:12.281514 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281479 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.281677 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281534 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.281677 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281559 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.281677 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.281677 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281620 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.281891 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.281679 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qgkcv\" (UniqueName: \"kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv\") pod \"24086d26-855b-4c41-baac-5c197dfef948\" (UID: \"24086d26-855b-4c41-baac-5c197dfef948\") " Jul 13 20:21:12.282031 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.282002 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config" (OuterVolumeSpecName: "console-config") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:21:12.282095 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.282048 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca" (OuterVolumeSpecName: "service-ca") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:21:12.282095 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.282051 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:21:12.284026 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.283999 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv" (OuterVolumeSpecName: "kube-api-access-qgkcv") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "kube-api-access-qgkcv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:21:12.284139 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.284117 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:21:12.284193 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.284174 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "24086d26-855b-4c41-baac-5c197dfef948" (UID: "24086d26-855b-4c41-baac-5c197dfef948"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:21:12.382792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382711 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-console-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:12.382792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382734 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-service-ca\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:12.382792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382769 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-oauth-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:12.382792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382784 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/24086d26-855b-4c41-baac-5c197dfef948-oauth-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:12.382792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382793 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qgkcv\" (UniqueName: \"kubernetes.io/projected/24086d26-855b-4c41-baac-5c197dfef948-kube-api-access-qgkcv\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:12.383013 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:12.382802 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/24086d26-855b-4c41-baac-5c197dfef948-console-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:21:13.019762 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:13.019706 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-85d8fb575f-qm8bp" event={"ID":"24086d26-855b-4c41-baac-5c197dfef948","Type":"ContainerDied","Data":"9c4a9c48025fbe8285186cf238af09fb640498f4c9d02fbb954a52d663444077"} Jul 13 20:21:13.019762 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:13.019765 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-85d8fb575f-qm8bp" Jul 13 20:21:13.020273 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:13.019789 2580 scope.go:117] "RemoveContainer" containerID="01e892860714c0c68b766b226e040fc452cbbc194ce689aecb3859c2237fa364" Jul 13 20:21:13.035443 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:13.035419 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:21:13.039586 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:13.039565 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-85d8fb575f-qm8bp"] Jul 13 20:21:14.343111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:21:14.343082 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="24086d26-855b-4c41-baac-5c197dfef948" path="/var/lib/kubelet/pods/24086d26-855b-4c41-baac-5c197dfef948/volumes" Jul 13 20:22:11.811832 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.811736 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-55cf956d57-fd8ht"] Jul 13 20:22:11.812259 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.812052 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="24086d26-855b-4c41-baac-5c197dfef948" containerName="console" Jul 13 20:22:11.812259 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.812063 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="24086d26-855b-4c41-baac-5c197dfef948" containerName="console" Jul 13 20:22:11.812259 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.812127 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="24086d26-855b-4c41-baac-5c197dfef948" containerName="console" Jul 13 20:22:11.814225 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.814205 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.819565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.819534 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 13 20:22:11.819795 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.819776 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 13 20:22:11.820049 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.820029 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vvhpf\"" Jul 13 20:22:11.820239 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.820222 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 13 20:22:11.820467 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.820445 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 13 20:22:11.821110 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.821078 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 13 20:22:11.826063 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.826046 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 13 20:22:11.833194 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.833174 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55cf956d57-fd8ht"] Jul 13 20:22:11.938840 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.938808 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-metrics-client-ca\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.938999 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.938850 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.938999 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.938883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.938999 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.938956 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5kxph\" (UniqueName: \"kubernetes.io/projected/eb095fcf-ff96-4854-9270-ee06adc81cb9-kube-api-access-5kxph\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.939130 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.938997 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.939130 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.939023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.939130 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.939086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-federate-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:11.939130 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:11.939117 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-serving-certs-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039608 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039566 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5kxph\" (UniqueName: \"kubernetes.io/projected/eb095fcf-ff96-4854-9270-ee06adc81cb9-kube-api-access-5kxph\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039608 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-federate-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039684 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-serving-certs-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-metrics-client-ca\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.039882 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.039845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.040552 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.040499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-serving-certs-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.040769 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.040619 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.040851 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.040803 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/eb095fcf-ff96-4854-9270-ee06adc81cb9-metrics-client-ca\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.042368 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.042340 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-federate-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.042485 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.042468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.042786 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.042769 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-telemeter-client-tls\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.042852 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.042781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/eb095fcf-ff96-4854-9270-ee06adc81cb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.047611 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.047590 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5kxph\" (UniqueName: \"kubernetes.io/projected/eb095fcf-ff96-4854-9270-ee06adc81cb9-kube-api-access-5kxph\") pod \"telemeter-client-55cf956d57-fd8ht\" (UID: \"eb095fcf-ff96-4854-9270-ee06adc81cb9\") " pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.124875 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.124804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" Jul 13 20:22:12.260098 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:12.260079 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-55cf956d57-fd8ht"] Jul 13 20:22:12.262647 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:22:12.262619 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb095fcf_ff96_4854_9270_ee06adc81cb9.slice/crio-a9e89ff0574c5aef652c601550a1d7e3d6dcf4ee0391d599a06ee8e4562d303a WatchSource:0}: Error finding container a9e89ff0574c5aef652c601550a1d7e3d6dcf4ee0391d599a06ee8e4562d303a: Status 404 returned error can't find the container with id a9e89ff0574c5aef652c601550a1d7e3d6dcf4ee0391d599a06ee8e4562d303a Jul 13 20:22:13.250243 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:13.250195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" event={"ID":"eb095fcf-ff96-4854-9270-ee06adc81cb9","Type":"ContainerStarted","Data":"a9e89ff0574c5aef652c601550a1d7e3d6dcf4ee0391d599a06ee8e4562d303a"} Jul 13 20:22:14.254989 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:14.254958 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" event={"ID":"eb095fcf-ff96-4854-9270-ee06adc81cb9","Type":"ContainerStarted","Data":"b3b70a06885b9490aa1cccbec944a2c6d8ed3697983a2dc1e6df2af081142e03"} Jul 13 20:22:15.260433 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.260401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" event={"ID":"eb095fcf-ff96-4854-9270-ee06adc81cb9","Type":"ContainerStarted","Data":"22514da66080e4a955456027339f4556d95831f33d7efadc290fc29a6e497ccc"} Jul 13 20:22:15.260806 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.260451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" event={"ID":"eb095fcf-ff96-4854-9270-ee06adc81cb9","Type":"ContainerStarted","Data":"59361fd9639f4b31266a428ac583c4a362f056504623c5fde263b4e78d8eb248"} Jul 13 20:22:15.282678 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.282634 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-55cf956d57-fd8ht" podStartSLOduration=2.404807441 podStartE2EDuration="4.282618506s" totalImagesPullingTime="1.877811065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:15.280028208 +0000 UTC m=+181.444587558" watchObservedRunningTime="2026-07-13 20:22:15.282618506 +0000 UTC m=+181.447177857" Jul 13 20:22:15.958330 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.958288 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:22:15.960635 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.960619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:15.972930 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:15.972909 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:22:16.074342 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5vfx\" (UniqueName: \"kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074342 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074343 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074542 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074379 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074542 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074464 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074542 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074492 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074542 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074510 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.074542 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.074535 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175048 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175011 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175091 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175178 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175384 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r5vfx\" (UniqueName: \"kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175384 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175384 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175319 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.175914 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.175890 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.177687 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.176411 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.177867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.176975 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.177867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.177628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.178538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.178513 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.178825 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.178803 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.184881 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.184861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5vfx\" (UniqueName: \"kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx\") pod \"console-6bcb7f5f6d-526tp\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.271188 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.271111 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:16.413668 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:16.413638 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:22:16.417004 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:22:16.416972 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda52c223a_8ef2_4444_abf1_3621707926a6.slice/crio-311f75bfcb06033c9a291d115688635e68457addcf72c8a55f91c5dd39b4247d WatchSource:0}: Error finding container 311f75bfcb06033c9a291d115688635e68457addcf72c8a55f91c5dd39b4247d: Status 404 returned error can't find the container with id 311f75bfcb06033c9a291d115688635e68457addcf72c8a55f91c5dd39b4247d Jul 13 20:22:17.268931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:17.268896 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcb7f5f6d-526tp" event={"ID":"a52c223a-8ef2-4444-abf1-3621707926a6","Type":"ContainerStarted","Data":"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632"} Jul 13 20:22:17.269164 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:17.268946 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcb7f5f6d-526tp" event={"ID":"a52c223a-8ef2-4444-abf1-3621707926a6","Type":"ContainerStarted","Data":"311f75bfcb06033c9a291d115688635e68457addcf72c8a55f91c5dd39b4247d"} Jul 13 20:22:17.306326 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:17.306275 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bcb7f5f6d-526tp" podStartSLOduration=2.306258562 podStartE2EDuration="2.306258562s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:17.304632763 +0000 UTC m=+183.469192140" watchObservedRunningTime="2026-07-13 20:22:17.306258562 +0000 UTC m=+183.470817913" Jul 13 20:22:26.271488 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.271448 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:26.271902 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.271505 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:26.276211 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.276190 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:26.309015 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.308989 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:22:26.359496 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.359473 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:22:26.514835 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.514808 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:22:26.518616 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.518599 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.541759 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.541712 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:22:26.667420 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667420 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667471 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fvqxq\" (UniqueName: \"kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.667601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.667598 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768280 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768284 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768304 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768326 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fvqxq\" (UniqueName: \"kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.768645 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.768448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.769141 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.769069 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.769141 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.769086 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.769141 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.769184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.769492 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.769274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.770908 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.770882 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.771050 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.771034 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.778304 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.778258 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fvqxq\" (UniqueName: \"kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq\") pod \"console-698d9f74f6-df5sk\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.827802 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.827765 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:26.961666 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:26.961643 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:22:26.964469 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:22:26.964427 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod914d14ce_7066_4be6_92ff_e22a9e3f9087.slice/crio-0c8e67296cefd1c430c8e1f4e5b827c01e645eb269e07797bca93595017534df WatchSource:0}: Error finding container 0c8e67296cefd1c430c8e1f4e5b827c01e645eb269e07797bca93595017534df: Status 404 returned error can't find the container with id 0c8e67296cefd1c430c8e1f4e5b827c01e645eb269e07797bca93595017534df Jul 13 20:22:27.311058 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:27.311021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698d9f74f6-df5sk" event={"ID":"914d14ce-7066-4be6-92ff-e22a9e3f9087","Type":"ContainerStarted","Data":"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11"} Jul 13 20:22:27.311438 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:27.311071 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698d9f74f6-df5sk" event={"ID":"914d14ce-7066-4be6-92ff-e22a9e3f9087","Type":"ContainerStarted","Data":"0c8e67296cefd1c430c8e1f4e5b827c01e645eb269e07797bca93595017534df"} Jul 13 20:22:27.330100 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:27.330054 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-698d9f74f6-df5sk" podStartSLOduration=1.33003769 podStartE2EDuration="1.33003769s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:22:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:22:27.329227124 +0000 UTC m=+193.493786472" watchObservedRunningTime="2026-07-13 20:22:27.33003769 +0000 UTC m=+193.494597040" Jul 13 20:22:36.828637 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:36.828609 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:36.829103 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:36.828692 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:36.833281 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:36.833263 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:37.357725 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:37.357696 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:22:37.408232 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:37.408203 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:22:51.382554 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.382490 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-554cbc8db6-szf9z" podUID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" containerName="console" containerID="cri-o://fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16" gracePeriod=15 Jul 13 20:22:51.630403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.630377 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:22:51.674338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674261 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674299 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xm598\" (UniqueName: \"kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674326 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674579 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674442 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674579 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674495 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674579 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674524 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674579 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674574 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert\") pod \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\" (UID: \"93f8f81b-4375-47b9-a9a0-e3ae524577c6\") " Jul 13 20:22:51.674826 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674703 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config" (OuterVolumeSpecName: "console-config") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:51.674826 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674785 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca" (OuterVolumeSpecName: "service-ca") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:51.674929 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674908 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.674929 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674926 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-service-ca\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.675198 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.674950 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:51.675198 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.675050 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:22:51.676561 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.676537 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:22:51.676660 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.676551 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598" (OuterVolumeSpecName: "kube-api-access-xm598") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "kube-api-access-xm598". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:22:51.676660 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.676609 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "93f8f81b-4375-47b9-a9a0-e3ae524577c6" (UID: "93f8f81b-4375-47b9-a9a0-e3ae524577c6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:22:51.775305 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.775269 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-oauth-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.775305 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.775299 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xm598\" (UniqueName: \"kubernetes.io/projected/93f8f81b-4375-47b9-a9a0-e3ae524577c6-kube-api-access-xm598\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.775305 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.775309 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-oauth-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.775527 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.775318 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/93f8f81b-4375-47b9-a9a0-e3ae524577c6-console-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:51.775527 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:51.775326 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/93f8f81b-4375-47b9-a9a0-e3ae524577c6-trusted-ca-bundle\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:22:52.413368 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.413330 2580 generic.go:356] "Generic (PLEG): container finished" podID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" containerID="fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16" exitCode=0 Jul 13 20:22:52.413867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.413409 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-554cbc8db6-szf9z" Jul 13 20:22:52.413867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.413430 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554cbc8db6-szf9z" event={"ID":"93f8f81b-4375-47b9-a9a0-e3ae524577c6","Type":"ContainerDied","Data":"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16"} Jul 13 20:22:52.413867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.413497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-554cbc8db6-szf9z" event={"ID":"93f8f81b-4375-47b9-a9a0-e3ae524577c6","Type":"ContainerDied","Data":"752f264a82b75edaaf0b6a7ab7e5d126ecbd1fee8335cb10b6f1eab7d7fbd930"} Jul 13 20:22:52.413867 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.413532 2580 scope.go:117] "RemoveContainer" containerID="fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16" Jul 13 20:22:52.421914 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.421896 2580 scope.go:117] "RemoveContainer" containerID="fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16" Jul 13 20:22:52.422181 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:22:52.422162 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16\": container with ID starting with fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16 not found: ID does not exist" containerID="fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16" Jul 13 20:22:52.422235 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.422192 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16"} err="failed to get container status \"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16\": rpc error: code = NotFound desc = could not find container \"fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16\": container with ID starting with fc6bfebcf98b76aaf11ccaa89c811c948fabb2d3cfed158d63882a84f61c9a16 not found: ID does not exist" Jul 13 20:22:52.429908 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.429887 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:22:52.433902 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:52.433882 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-554cbc8db6-szf9z"] Jul 13 20:22:54.345428 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:22:54.345395 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" path="/var/lib/kubelet/pods/93f8f81b-4375-47b9-a9a0-e3ae524577c6/volumes" Jul 13 20:23:02.436124 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.436084 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-6bcb7f5f6d-526tp" podUID="a52c223a-8ef2-4444-abf1-3621707926a6" containerName="console" containerID="cri-o://029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632" gracePeriod=15 Jul 13 20:23:02.678783 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.678759 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:23:02.759010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.758933 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759010 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759009 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759228 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759111 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759228 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759188 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759228 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759217 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759371 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759254 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759371 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759289 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5vfx\" (UniqueName: \"kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx\") pod \"a52c223a-8ef2-4444-abf1-3621707926a6\" (UID: \"a52c223a-8ef2-4444-abf1-3621707926a6\") " Jul 13 20:23:02.759371 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759339 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:23:02.759514 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759386 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:23:02.759626 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759594 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca" (OuterVolumeSpecName: "service-ca") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:23:02.759684 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759649 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config" (OuterVolumeSpecName: "console-config") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:23:02.759726 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759666 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-oauth-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.759726 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.759707 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-trusted-ca-bundle\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.761412 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.761380 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:23:02.761484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.761416 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx" (OuterVolumeSpecName: "kube-api-access-r5vfx") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "kube-api-access-r5vfx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:23:02.761484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.761473 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a52c223a-8ef2-4444-abf1-3621707926a6" (UID: "a52c223a-8ef2-4444-abf1-3621707926a6"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:23:02.860820 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.860792 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-oauth-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.860820 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.860816 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-service-ca\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.860820 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.860825 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a52c223a-8ef2-4444-abf1-3621707926a6-console-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.861003 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.860834 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a52c223a-8ef2-4444-abf1-3621707926a6-console-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:02.861003 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:02.860842 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5vfx\" (UniqueName: \"kubernetes.io/projected/a52c223a-8ef2-4444-abf1-3621707926a6-kube-api-access-r5vfx\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:23:03.458217 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.458178 2580 generic.go:356] "Generic (PLEG): container finished" podID="a52c223a-8ef2-4444-abf1-3621707926a6" containerID="029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632" exitCode=0 Jul 13 20:23:03.458607 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.458221 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bcb7f5f6d-526tp" Jul 13 20:23:03.458607 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.458223 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcb7f5f6d-526tp" event={"ID":"a52c223a-8ef2-4444-abf1-3621707926a6","Type":"ContainerDied","Data":"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632"} Jul 13 20:23:03.458607 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.458348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bcb7f5f6d-526tp" event={"ID":"a52c223a-8ef2-4444-abf1-3621707926a6","Type":"ContainerDied","Data":"311f75bfcb06033c9a291d115688635e68457addcf72c8a55f91c5dd39b4247d"} Jul 13 20:23:03.458607 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.458371 2580 scope.go:117] "RemoveContainer" containerID="029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632" Jul 13 20:23:03.467453 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.467436 2580 scope.go:117] "RemoveContainer" containerID="029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632" Jul 13 20:23:03.467696 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:23:03.467677 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632\": container with ID starting with 029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632 not found: ID does not exist" containerID="029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632" Jul 13 20:23:03.467788 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.467709 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632"} err="failed to get container status \"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632\": rpc error: code = NotFound desc = could not find container \"029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632\": container with ID starting with 029ebc443cffd7b656c05da098438f7e871fd51e8ce68925f6157263f807c632 not found: ID does not exist" Jul 13 20:23:03.478005 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.477977 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:23:03.481552 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:03.481531 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-6bcb7f5f6d-526tp"] Jul 13 20:23:04.339144 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:04.339114 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a52c223a-8ef2-4444-abf1-3621707926a6" path="/var/lib/kubelet/pods/a52c223a-8ef2-4444-abf1-3621707926a6/volumes" Jul 13 20:23:59.277492 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277456 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq"] Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277771 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" containerName="console" Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277783 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" containerName="console" Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277799 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a52c223a-8ef2-4444-abf1-3621707926a6" containerName="console" Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277805 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a52c223a-8ef2-4444-abf1-3621707926a6" containerName="console" Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277866 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a52c223a-8ef2-4444-abf1-3621707926a6" containerName="console" Jul 13 20:23:59.277931 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.277875 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="93f8f81b-4375-47b9-a9a0-e3ae524577c6" containerName="console" Jul 13 20:23:59.281040 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.281024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.284362 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.284342 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-5hzm4\"" Jul 13 20:23:59.284516 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.284498 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 13 20:23:59.284587 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.284540 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 13 20:23:59.294800 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.294782 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq"] Jul 13 20:23:59.394565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.394519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.394565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.394569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.394792 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.394589 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d5p9m\" (UniqueName: \"kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.496058 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.496032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.496185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.496065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.496185 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.496087 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d5p9m\" (UniqueName: \"kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.496430 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.496409 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.496491 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.496466 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.505949 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.505924 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d5p9m\" (UniqueName: \"kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m\") pod \"320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.590105 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.590082 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:23:59.724869 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:23:59.724729 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq"] Jul 13 20:23:59.727775 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:23:59.727730 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d2680b2_a997_47d1_a24b_cebbe330db3b.slice/crio-7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de WatchSource:0}: Error finding container 7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de: Status 404 returned error can't find the container with id 7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de Jul 13 20:24:00.674334 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:00.674297 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" event={"ID":"1d2680b2-a997-47d1-a24b-cebbe330db3b","Type":"ContainerStarted","Data":"7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de"} Jul 13 20:24:05.697971 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:05.697936 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerID="e0c38132ecd5743ba4914e49664ff926bac07444a85ce7c0b72010b56a814ba4" exitCode=0 Jul 13 20:24:05.698334 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:05.698010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" event={"ID":"1d2680b2-a997-47d1-a24b-cebbe330db3b","Type":"ContainerDied","Data":"e0c38132ecd5743ba4914e49664ff926bac07444a85ce7c0b72010b56a814ba4"} Jul 13 20:24:08.713075 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:08.713040 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerID="24c3482f49a6d78516a5a83c46653232ac83ea0d2d208e710622786368e91054" exitCode=0 Jul 13 20:24:08.713075 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:08.713079 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" event={"ID":"1d2680b2-a997-47d1-a24b-cebbe330db3b","Type":"ContainerDied","Data":"24c3482f49a6d78516a5a83c46653232ac83ea0d2d208e710622786368e91054"} Jul 13 20:24:15.757685 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:15.757641 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:24:15.760088 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:15.758189 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:24:15.763048 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:15.763031 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 13 20:24:16.756674 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:16.756642 2580 generic.go:356] "Generic (PLEG): container finished" podID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerID="50b8bcc25c6774111fe30f499a0eb5f9da1251dbed9ad02d4d052f6bf72e6c60" exitCode=0 Jul 13 20:24:16.756853 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:16.756680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" event={"ID":"1d2680b2-a997-47d1-a24b-cebbe330db3b","Type":"ContainerDied","Data":"50b8bcc25c6774111fe30f499a0eb5f9da1251dbed9ad02d4d052f6bf72e6c60"} Jul 13 20:24:17.889708 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.889684 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:24:17.958651 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.958628 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle\") pod \"1d2680b2-a997-47d1-a24b-cebbe330db3b\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " Jul 13 20:24:17.958816 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.958721 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util\") pod \"1d2680b2-a997-47d1-a24b-cebbe330db3b\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " Jul 13 20:24:17.958816 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.958766 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d5p9m\" (UniqueName: \"kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m\") pod \"1d2680b2-a997-47d1-a24b-cebbe330db3b\" (UID: \"1d2680b2-a997-47d1-a24b-cebbe330db3b\") " Jul 13 20:24:17.959292 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.959265 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle" (OuterVolumeSpecName: "bundle") pod "1d2680b2-a997-47d1-a24b-cebbe330db3b" (UID: "1d2680b2-a997-47d1-a24b-cebbe330db3b"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:24:17.961160 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.961131 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m" (OuterVolumeSpecName: "kube-api-access-d5p9m") pod "1d2680b2-a997-47d1-a24b-cebbe330db3b" (UID: "1d2680b2-a997-47d1-a24b-cebbe330db3b"). InnerVolumeSpecName "kube-api-access-d5p9m". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:24:17.963421 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:17.963402 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util" (OuterVolumeSpecName: "util") pod "1d2680b2-a997-47d1-a24b-cebbe330db3b" (UID: "1d2680b2-a997-47d1-a24b-cebbe330db3b"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:24:18.059636 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.059614 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-util\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:24:18.059718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.059636 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d5p9m\" (UniqueName: \"kubernetes.io/projected/1d2680b2-a997-47d1-a24b-cebbe330db3b-kube-api-access-d5p9m\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:24:18.059718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.059646 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/1d2680b2-a997-47d1-a24b-cebbe330db3b-bundle\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:24:18.766332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.766249 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" event={"ID":"1d2680b2-a997-47d1-a24b-cebbe330db3b","Type":"ContainerDied","Data":"7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de"} Jul 13 20:24:18.766332 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.766291 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/320da698f62953fa8a11a45982d6c7ee4dcf44b832e3de43010dcb42b5l7qpq" Jul 13 20:24:18.766489 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:18.766338 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7ab2c324f2794960c1d250d4c27470fbfc4b78a25a094aeac9b437a457a2d9de" Jul 13 20:24:27.715274 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715239 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-77f648d694-9dcxv"] Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715546 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="pull" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715556 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="pull" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715568 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="util" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715573 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="util" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715590 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="extract" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715596 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="extract" Jul 13 20:24:27.715716 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.715643 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="1d2680b2-a997-47d1-a24b-cebbe330db3b" containerName="extract" Jul 13 20:24:27.719335 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.719318 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.722106 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722081 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 13 20:24:27.722366 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722349 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 13 20:24:27.722479 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722461 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 13 20:24:27.722540 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722477 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-qzpk6\"" Jul 13 20:24:27.722681 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722669 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 13 20:24:27.722718 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.722704 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 13 20:24:27.741967 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.741946 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-9dcxv"] Jul 13 20:24:27.837759 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.837712 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.837895 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.837824 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ckv9x\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-kube-api-access-ckv9x\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.837895 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.837860 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/76005260-f387-495e-85dc-a3b82e041cc6-cabundle0\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.866565 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.866533 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2"] Jul 13 20:24:27.868916 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.868900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:27.871866 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.871847 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 13 20:24:27.881448 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.881429 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2"] Jul 13 20:24:27.938834 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.938809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ckv9x\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-kube-api-access-ckv9x\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.938934 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.938845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/76005260-f387-495e-85dc-a3b82e041cc6-cabundle0\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.938934 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.938890 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:27.939003 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.938945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.939041 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.939025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8410316a-5973-471d-ac04-bb2cc23c28fc-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:27.939073 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:27.939056 2580 secret.go:281] references non-existent secret key: ca.crt Jul 13 20:24:27.939104 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:27.939072 2580 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 13 20:24:27.939104 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:27.939083 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-77f648d694-9dcxv: references non-existent secret key: ca.crt Jul 13 20:24:27.939172 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.939052 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pqqgj\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-kube-api-access-pqqgj\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:27.939172 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:27.939143 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates podName:76005260-f387-495e-85dc-a3b82e041cc6 nodeName:}" failed. No retries permitted until 2026-07-13 20:24:28.439128616 +0000 UTC m=+314.603687944 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates") pod "keda-operator-77f648d694-9dcxv" (UID: "76005260-f387-495e-85dc-a3b82e041cc6") : references non-existent secret key: ca.crt Jul 13 20:24:27.939436 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.939419 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/76005260-f387-495e-85dc-a3b82e041cc6-cabundle0\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:27.949471 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:27.949445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ckv9x\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-kube-api-access-ckv9x\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:28.039682 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.039628 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8410316a-5973-471d-ac04-bb2cc23c28fc-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.039682 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.039654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pqqgj\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-kube-api-access-pqqgj\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.039839 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.039687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.039839 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:28.039801 2580 secret.go:281] references non-existent secret key: tls.crt Jul 13 20:24:28.039839 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:28.039812 2580 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 13 20:24:28.039839 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:28.039826 2580 projected.go:266] Couldn't get secret openshift-keda/keda-metrics-apiserver-certs: secret "keda-metrics-apiserver-certs" not found Jul 13 20:24:28.039955 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:28.039843 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2: [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 13 20:24:28.039955 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:24:28.039900 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates podName:8410316a-5973-471d-ac04-bb2cc23c28fc nodeName:}" failed. No retries permitted until 2026-07-13 20:24:28.539883069 +0000 UTC m=+314.704442401 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates") pod "keda-metrics-apiserver-55557df8c-cxbl2" (UID: "8410316a-5973-471d-ac04-bb2cc23c28fc") : [references non-existent secret key: tls.crt, secret "keda-metrics-apiserver-certs" not found] Jul 13 20:24:28.040072 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.040056 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/8410316a-5973-471d-ac04-bb2cc23c28fc-temp-vol\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.052014 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.051989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pqqgj\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-kube-api-access-pqqgj\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.443172 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.443141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:28.445735 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.445709 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/76005260-f387-495e-85dc-a3b82e041cc6-certificates\") pod \"keda-operator-77f648d694-9dcxv\" (UID: \"76005260-f387-495e-85dc-a3b82e041cc6\") " pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:28.544383 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.544347 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.547417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.547387 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/8410316a-5973-471d-ac04-bb2cc23c28fc-certificates\") pod \"keda-metrics-apiserver-55557df8c-cxbl2\" (UID: \"8410316a-5973-471d-ac04-bb2cc23c28fc\") " pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.629451 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.629412 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:28.779706 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.779519 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:28.788582 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.788535 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-77f648d694-9dcxv"] Jul 13 20:24:28.791377 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:24:28.791350 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76005260_f387_495e_85dc_a3b82e041cc6.slice/crio-bd20845deb288a7eadc84f85ee2ae685476928b87eaaa07dc8b3478ef8d6409b WatchSource:0}: Error finding container bd20845deb288a7eadc84f85ee2ae685476928b87eaaa07dc8b3478ef8d6409b: Status 404 returned error can't find the container with id bd20845deb288a7eadc84f85ee2ae685476928b87eaaa07dc8b3478ef8d6409b Jul 13 20:24:28.792707 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.792686 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 13 20:24:28.808497 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.808466 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-9dcxv" event={"ID":"76005260-f387-495e-85dc-a3b82e041cc6","Type":"ContainerStarted","Data":"bd20845deb288a7eadc84f85ee2ae685476928b87eaaa07dc8b3478ef8d6409b"} Jul 13 20:24:28.923999 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:28.923921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2"] Jul 13 20:24:28.928079 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:24:28.928053 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8410316a_5973_471d_ac04_bb2cc23c28fc.slice/crio-c458406decd0c0bee24a9c92047b100d9657fcd199bd216b3bc6f872cc73b7e7 WatchSource:0}: Error finding container c458406decd0c0bee24a9c92047b100d9657fcd199bd216b3bc6f872cc73b7e7: Status 404 returned error can't find the container with id c458406decd0c0bee24a9c92047b100d9657fcd199bd216b3bc6f872cc73b7e7 Jul 13 20:24:29.815444 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:29.815400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" event={"ID":"8410316a-5973-471d-ac04-bb2cc23c28fc","Type":"ContainerStarted","Data":"c458406decd0c0bee24a9c92047b100d9657fcd199bd216b3bc6f872cc73b7e7"} Jul 13 20:24:32.830588 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.830548 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" event={"ID":"8410316a-5973-471d-ac04-bb2cc23c28fc","Type":"ContainerStarted","Data":"94364aa93be5ab781da3dff62339881010f2888845e168076b0c7be284669c8f"} Jul 13 20:24:32.831066 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.830689 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:32.832152 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.832129 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-77f648d694-9dcxv" event={"ID":"76005260-f387-495e-85dc-a3b82e041cc6","Type":"ContainerStarted","Data":"f023573a574f15d22ee3b0a20342e6e517df55714eb8a0ccc8304b7614b0c27c"} Jul 13 20:24:32.832283 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.832254 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:24:32.868719 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.868646 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" podStartSLOduration=2.182389267 podStartE2EDuration="5.868631942s" totalImagesPullingTime="3.686242675s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:32.866906129 +0000 UTC m=+319.031465479" watchObservedRunningTime="2026-07-13 20:24:32.868631942 +0000 UTC m=+319.033191292" Jul 13 20:24:32.939282 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:32.939229 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-77f648d694-9dcxv" podStartSLOduration=2.111405087 podStartE2EDuration="5.93920813s" totalImagesPullingTime="3.827803043s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:24:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:24:32.938901275 +0000 UTC m=+319.103460628" watchObservedRunningTime="2026-07-13 20:24:32.93920813 +0000 UTC m=+319.103767481" Jul 13 20:24:43.840296 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:43.840269 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-55557df8c-cxbl2" Jul 13 20:24:53.838832 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:24:53.838796 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-77f648d694-9dcxv" Jul 13 20:25:37.816997 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.816923 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-cn5tg"] Jul 13 20:25:37.820041 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.820026 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:37.822398 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.822376 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 13 20:25:37.822570 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.822548 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-fdskb\"" Jul 13 20:25:37.822983 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.822967 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 13 20:25:37.822983 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.822978 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jul 13 20:25:37.832485 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.832463 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-cn5tg"] Jul 13 20:25:37.885605 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.885580 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7770607-d7c8-42a8-a416-1e63474c7fcd-cert\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:37.885722 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.885624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-shhdr\" (UniqueName: \"kubernetes.io/projected/d7770607-d7c8-42a8-a416-1e63474c7fcd-kube-api-access-shhdr\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:37.986924 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.986892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7770607-d7c8-42a8-a416-1e63474c7fcd-cert\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:37.987069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.986931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-shhdr\" (UniqueName: \"kubernetes.io/projected/d7770607-d7c8-42a8-a416-1e63474c7fcd-kube-api-access-shhdr\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:37.989448 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:37.989426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d7770607-d7c8-42a8-a416-1e63474c7fcd-cert\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:38.006898 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:38.006877 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-shhdr\" (UniqueName: \"kubernetes.io/projected/d7770607-d7c8-42a8-a416-1e63474c7fcd-kube-api-access-shhdr\") pod \"llmisvc-controller-manager-84fc959947-cn5tg\" (UID: \"d7770607-d7c8-42a8-a416-1e63474c7fcd\") " pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:38.130406 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:38.130336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:38.261901 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:38.261870 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-84fc959947-cn5tg"] Jul 13 20:25:38.265099 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:25:38.265065 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podd7770607_d7c8_42a8_a416_1e63474c7fcd.slice/crio-9020117b797b41c5d9cccb8cc14847224c63c9509939f338733bc9a774c39f60 WatchSource:0}: Error finding container 9020117b797b41c5d9cccb8cc14847224c63c9509939f338733bc9a774c39f60: Status 404 returned error can't find the container with id 9020117b797b41c5d9cccb8cc14847224c63c9509939f338733bc9a774c39f60 Jul 13 20:25:39.106521 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:39.106485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" event={"ID":"d7770607-d7c8-42a8-a416-1e63474c7fcd","Type":"ContainerStarted","Data":"9020117b797b41c5d9cccb8cc14847224c63c9509939f338733bc9a774c39f60"} Jul 13 20:25:42.121073 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:42.121035 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" event={"ID":"d7770607-d7c8-42a8-a416-1e63474c7fcd","Type":"ContainerStarted","Data":"8d9f0c5d9caa8dc877e9c2cfb2773e465923c4854908c94b093bc654e9477099"} Jul 13 20:25:42.121465 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:42.121132 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:25:42.161216 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:25:42.161166 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" podStartSLOduration=2.358590736 podStartE2EDuration="5.161149383s" totalImagesPullingTime="2.802558647s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:25:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:25:42.158788101 +0000 UTC m=+388.323347451" watchObservedRunningTime="2026-07-13 20:25:42.161149383 +0000 UTC m=+388.325708736" Jul 13 20:26:13.126872 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:26:13.126843 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-84fc959947-cn5tg" Jul 13 20:27:03.814820 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.814777 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/s3-init-4qp95"] Jul 13 20:27:03.821115 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.819336 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4qp95" Jul 13 20:27:03.822260 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.822229 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-j98tn\"" Jul 13 20:27:03.822377 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.822281 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jul 13 20:27:03.881767 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.881724 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-4qp95"] Jul 13 20:27:03.977287 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.977258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hkqvg\" (UniqueName: \"kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:03.977412 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:03.977345 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:04.078040 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.077973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:04.078040 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.078021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hkqvg\" (UniqueName: \"kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:04.078357 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.078337 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:04.091788 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.091764 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hkqvg\" (UniqueName: \"kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg\") pod \"s3-init-4qp95\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " pod="kserve/s3-init-4qp95" Jul 13 20:27:04.144065 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.144040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4qp95" Jul 13 20:27:04.296159 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.296134 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/s3-init-4qp95"] Jul 13 20:27:04.298674 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:27:04.298645 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod17e5cb49_23e6_48b8_99b3_96a815e59d38.slice/crio-9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689 WatchSource:0}: Error finding container 9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689: Status 404 returned error can't find the container with id 9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689 Jul 13 20:27:04.452003 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:04.451918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerStarted","Data":"9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689"} Jul 13 20:27:08.475733 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:08.475694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerStarted","Data":"c3847093b3e09dd7f40b8c4b90ab19d65dd10e9fcd2eaf950d558bbf337944b8"} Jul 13 20:27:09.481400 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:09.481315 2580 generic.go:356] "Generic (PLEG): container finished" podID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerID="c3847093b3e09dd7f40b8c4b90ab19d65dd10e9fcd2eaf950d558bbf337944b8" exitCode=0 Jul 13 20:27:09.481777 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:09.481405 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerDied","Data":"c3847093b3e09dd7f40b8c4b90ab19d65dd10e9fcd2eaf950d558bbf337944b8"} Jul 13 20:27:14.509394 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:14.509300 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerStarted","Data":"d12dcc1b2bbefe9b1c8ac111ac799db4435c4217b808ae4185cbae6a536b3b2c"} Jul 13 20:27:14.528032 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:14.527986 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/s3-init-4qp95" podStartSLOduration=2.144892071 podStartE2EDuration="11.527970982s" totalImagesPullingTime="8.205072112s" totalInitContainerRuntime="1.178006799s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:27:03 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:27:14.52635238 +0000 UTC m=+480.690911730" watchObservedRunningTime="2026-07-13 20:27:14.527970982 +0000 UTC m=+480.692530332" Jul 13 20:27:16.827814 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.827776 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-74857fddc6-lpbtg"] Jul 13 20:27:16.834278 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.834256 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.844685 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.844664 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74857fddc6-lpbtg"] Jul 13 20:27:16.996508 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996472 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-oauth-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996508 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996507 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2qcf2\" (UniqueName: \"kubernetes.io/projected/f4252770-8cfb-4f47-91e8-30e36230b268-kube-api-access-2qcf2\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996763 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-service-ca\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996763 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996548 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996763 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996642 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-console-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996896 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-oauth-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:16.996896 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:16.996811 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-trusted-ca-bundle\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097431 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-trusted-ca-bundle\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097431 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-oauth-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097431 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097403 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2qcf2\" (UniqueName: \"kubernetes.io/projected/f4252770-8cfb-4f47-91e8-30e36230b268-kube-api-access-2qcf2\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097736 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097524 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-service-ca\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097736 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097736 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-console-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.097736 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.097707 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-oauth-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.098499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.098292 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-service-ca\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.098499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.098347 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-trusted-ca-bundle\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.098499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.098422 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-console-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.098499 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.098445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f4252770-8cfb-4f47-91e8-30e36230b268-oauth-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.100157 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.100132 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-serving-cert\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.100363 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.100344 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f4252770-8cfb-4f47-91e8-30e36230b268-console-oauth-config\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.110764 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.110712 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2qcf2\" (UniqueName: \"kubernetes.io/projected/f4252770-8cfb-4f47-91e8-30e36230b268-kube-api-access-2qcf2\") pod \"console-74857fddc6-lpbtg\" (UID: \"f4252770-8cfb-4f47-91e8-30e36230b268\") " pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.143646 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.143619 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:17.292377 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.292345 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74857fddc6-lpbtg"] Jul 13 20:27:17.296161 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:27:17.296131 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4252770_8cfb_4f47_91e8_30e36230b268.slice/crio-cbff7bcf5fefbd1a22128fef805cc90114ad5dde7035aac8ecc2be366a99d8db WatchSource:0}: Error finding container cbff7bcf5fefbd1a22128fef805cc90114ad5dde7035aac8ecc2be366a99d8db: Status 404 returned error can't find the container with id cbff7bcf5fefbd1a22128fef805cc90114ad5dde7035aac8ecc2be366a99d8db Jul 13 20:27:17.525706 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.525670 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74857fddc6-lpbtg" event={"ID":"f4252770-8cfb-4f47-91e8-30e36230b268","Type":"ContainerStarted","Data":"5bed049c32588672f7fea49b6e9a6eceab0fe37ea42f1c660e0f01497c094152"} Jul 13 20:27:17.525915 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.525726 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74857fddc6-lpbtg" event={"ID":"f4252770-8cfb-4f47-91e8-30e36230b268","Type":"ContainerStarted","Data":"cbff7bcf5fefbd1a22128fef805cc90114ad5dde7035aac8ecc2be366a99d8db"} Jul 13 20:27:17.545502 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:17.545447 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-74857fddc6-lpbtg" podStartSLOduration=1.545426801 podStartE2EDuration="1.545426801s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:27:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:27:17.544924588 +0000 UTC m=+483.709483940" watchObservedRunningTime="2026-07-13 20:27:17.545426801 +0000 UTC m=+483.709986153" Jul 13 20:27:27.144115 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:27.144074 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:27.144577 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:27.144133 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:27.148346 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:27.148325 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:27.570014 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:27.569992 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74857fddc6-lpbtg" Jul 13 20:27:27.622056 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:27.622030 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:27:52.648377 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:52.648318 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-698d9f74f6-df5sk" podUID="914d14ce-7066-4be6-92ff-e22a9e3f9087" containerName="console" containerID="cri-o://4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11" gracePeriod=15 Jul 13 20:27:52.897441 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:52.897419 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:27:53.011170 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011091 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011170 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011126 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fvqxq\" (UniqueName: \"kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011170 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011150 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011184 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011214 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011294 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011315 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle\") pod \"914d14ce-7066-4be6-92ff-e22a9e3f9087\" (UID: \"914d14ce-7066-4be6-92ff-e22a9e3f9087\") " Jul 13 20:27:53.011793 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011703 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca" (OuterVolumeSpecName: "service-ca") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:27:53.011793 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011724 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config" (OuterVolumeSpecName: "console-config") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:27:53.011940 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011788 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:27:53.011940 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.011885 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:27:53.013463 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.013440 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq" (OuterVolumeSpecName: "kube-api-access-fvqxq") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "kube-api-access-fvqxq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:27:53.013546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.013537 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:27:53.013612 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.013591 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "914d14ce-7066-4be6-92ff-e22a9e3f9087" (UID: "914d14ce-7066-4be6-92ff-e22a9e3f9087"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:27:53.112709 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112677 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-oauth-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112709 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112700 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-trusted-ca-bundle\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112709 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112709 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-oauth-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112709 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112717 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fvqxq\" (UniqueName: \"kubernetes.io/projected/914d14ce-7066-4be6-92ff-e22a9e3f9087-kube-api-access-fvqxq\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112726 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112734 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/914d14ce-7066-4be6-92ff-e22a9e3f9087-service-ca\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.112962 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.112758 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/914d14ce-7066-4be6-92ff-e22a9e3f9087-console-serving-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:27:53.678975 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.678936 2580 generic.go:356] "Generic (PLEG): container finished" podID="914d14ce-7066-4be6-92ff-e22a9e3f9087" containerID="4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11" exitCode=0 Jul 13 20:27:53.679403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.678984 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-698d9f74f6-df5sk" Jul 13 20:27:53.679403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.679021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698d9f74f6-df5sk" event={"ID":"914d14ce-7066-4be6-92ff-e22a9e3f9087","Type":"ContainerDied","Data":"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11"} Jul 13 20:27:53.679403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.679072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-698d9f74f6-df5sk" event={"ID":"914d14ce-7066-4be6-92ff-e22a9e3f9087","Type":"ContainerDied","Data":"0c8e67296cefd1c430c8e1f4e5b827c01e645eb269e07797bca93595017534df"} Jul 13 20:27:53.679403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.679125 2580 scope.go:117] "RemoveContainer" containerID="4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11" Jul 13 20:27:53.690927 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.690897 2580 scope.go:117] "RemoveContainer" containerID="4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11" Jul 13 20:27:53.691389 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:27:53.691214 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11\": container with ID starting with 4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11 not found: ID does not exist" containerID="4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11" Jul 13 20:27:53.691389 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.691242 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11"} err="failed to get container status \"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11\": rpc error: code = NotFound desc = could not find container \"4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11\": container with ID starting with 4fcf6932d4992312d3d6478cf18eea8ddfe022b30e10e7d215f7e5ca49afdc11 not found: ID does not exist" Jul 13 20:27:53.706641 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.706616 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:27:53.713417 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:53.713398 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-698d9f74f6-df5sk"] Jul 13 20:27:54.340061 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:27:54.340029 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="914d14ce-7066-4be6-92ff-e22a9e3f9087" path="/var/lib/kubelet/pods/914d14ce-7066-4be6-92ff-e22a9e3f9087/volumes" Jul 13 20:28:19.800759 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:19.800702 2580 generic.go:356] "Generic (PLEG): container finished" podID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerID="d12dcc1b2bbefe9b1c8ac111ac799db4435c4217b808ae4185cbae6a536b3b2c" exitCode=0 Jul 13 20:28:19.801210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:19.800782 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerDied","Data":"d12dcc1b2bbefe9b1c8ac111ac799db4435c4217b808ae4185cbae6a536b3b2c"} Jul 13 20:28:20.932487 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:20.932457 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4qp95" Jul 13 20:28:21.041214 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.041184 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache\") pod \"17e5cb49-23e6-48b8-99b3-96a815e59d38\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " Jul 13 20:28:21.041375 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.041255 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hkqvg\" (UniqueName: \"kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg\") pod \"17e5cb49-23e6-48b8-99b3-96a815e59d38\" (UID: \"17e5cb49-23e6-48b8-99b3-96a815e59d38\") " Jul 13 20:28:21.043538 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.043511 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg" (OuterVolumeSpecName: "kube-api-access-hkqvg") pod "17e5cb49-23e6-48b8-99b3-96a815e59d38" (UID: "17e5cb49-23e6-48b8-99b3-96a815e59d38"). InnerVolumeSpecName "kube-api-access-hkqvg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:28:21.059778 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.059706 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache" (OuterVolumeSpecName: "model-cache") pod "17e5cb49-23e6-48b8-99b3-96a815e59d38" (UID: "17e5cb49-23e6-48b8-99b3-96a815e59d38"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:28:21.141753 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.141709 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/17e5cb49-23e6-48b8-99b3-96a815e59d38-model-cache\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:28:21.141753 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.141765 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hkqvg\" (UniqueName: \"kubernetes.io/projected/17e5cb49-23e6-48b8-99b3-96a815e59d38-kube-api-access-hkqvg\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:28:21.810390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.810348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/s3-init-4qp95" event={"ID":"17e5cb49-23e6-48b8-99b3-96a815e59d38","Type":"ContainerDied","Data":"9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689"} Jul 13 20:28:21.810390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.810366 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/s3-init-4qp95" Jul 13 20:28:21.810390 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:21.810386 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9285e6b017ff5b327151f1bb232a04870408d2bd1fdd81a2e05efe6a3011c689" Jul 13 20:28:33.924283 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924249 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924582 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerName="s3-init" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924593 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerName="s3-init" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924614 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="914d14ce-7066-4be6-92ff-e22a9e3f9087" containerName="console" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924619 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="914d14ce-7066-4be6-92ff-e22a9e3f9087" containerName="console" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924631 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerName="download-hf-model" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924638 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerName="download-hf-model" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924684 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17e5cb49-23e6-48b8-99b3-96a815e59d38" containerName="s3-init" Jul 13 20:28:33.924697 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.924694 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="914d14ce-7066-4be6-92ff-e22a9e3f9087" containerName="console" Jul 13 20:28:33.928162 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.928142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:33.930513 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.930491 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-predictor-serving-cert\"" Jul 13 20:28:33.931076 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931056 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-kvq55\"" Jul 13 20:28:33.931171 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931035 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"odh-kserve-custom-ca-bundle\"" Jul 13 20:28:33.931171 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931061 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 13 20:28:33.931324 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931309 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-1-kube-rbac-proxy-sar-config\"" Jul 13 20:28:33.931389 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931332 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"seaweedfs-s3-creds\"" Jul 13 20:28:33.931389 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.931359 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"kube-root-ca.crt\"" Jul 13 20:28:33.944016 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:33.943993 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:28:34.057564 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.057527 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.057702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.057574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.057702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.057629 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.057702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.057666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.057835 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.057707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f2jxg\" (UniqueName: \"kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159178 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159338 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f2jxg\" (UniqueName: \"kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159700 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159672 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.159979 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.159952 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.160127 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.160109 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.162086 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.162059 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.170301 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.170281 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f2jxg\" (UniqueName: \"kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg\") pod \"isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.239167 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.239098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:28:34.363387 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.363359 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:28:34.367938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.367910 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.370646 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.370625 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-predictor-serving-cert\"" Jul 13 20:28:34.370646 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.370635 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-xgboost-graph-kube-rbac-proxy-sar-config\"" Jul 13 20:28:34.377481 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.377459 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:28:34.381936 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.381916 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:28:34.384286 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:28:34.384262 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod811dd1e2_0d2a_4aab_8a91_be98ac0f7946.slice/crio-88d78d69568e08bd9625b6237b9608c01d3d37b5cc66bbe5d21b14f98beb3b71 WatchSource:0}: Error finding container 88d78d69568e08bd9625b6237b9608c01d3d37b5cc66bbe5d21b14f98beb3b71: Status 404 returned error can't find the container with id 88d78d69568e08bd9625b6237b9608c01d3d37b5cc66bbe5d21b14f98beb3b71 Jul 13 20:28:34.462633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.462604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9mvbv\" (UniqueName: \"kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.462811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.462670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.462811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.462709 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.462811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.462758 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.462811 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.462803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.517722 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.517651 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:28:34.521201 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.521185 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.523387 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.523370 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-kube-rbac-proxy-sar-config\"" Jul 13 20:28:34.523387 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.523377 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"isvc-sklearn-graph-2-predictor-serving-cert\"" Jul 13 20:28:34.530861 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.530841 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:28:34.564064 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564200 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564072 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564200 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564096 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564200 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564129 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564200 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9mvbv\" (UniqueName: \"kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564378 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:28:34.564206 2580 secret.go:189] Couldn't get secret kserve-ci-e2e-test/isvc-xgboost-graph-predictor-serving-cert: secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 13 20:28:34.564378 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:28:34.564285 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls podName:98a6fde5-c82d-4a6d-a057-fdc3399a2025 nodeName:}" failed. No retries permitted until 2026-07-13 20:28:35.064263832 +0000 UTC m=+561.228823166 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "proxy-tls" (UniqueName: "kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls") pod "isvc-xgboost-graph-predictor-555fc69d56-xcfvp" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025") : secret "isvc-xgboost-graph-predictor-serving-cert" not found Jul 13 20:28:34.564610 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564586 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564799 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564779 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.564844 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.564836 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.573210 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.573186 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9mvbv\" (UniqueName: \"kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:34.665001 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.664960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.665197 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.665086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.665197 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.665122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.665197 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.665180 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-44d29\" (UniqueName: \"kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.665323 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.665251 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.765702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.765664 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-44d29\" (UniqueName: \"kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.765702 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.765708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.765938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.765847 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.765998 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.765953 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.765998 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.765981 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.766277 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.766252 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.766723 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.766704 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.766808 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.766728 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.768369 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.768321 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.774441 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.774416 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-44d29\" (UniqueName: \"kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29\") pod \"isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.834055 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.834025 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:28:34.865302 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.865258 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerStarted","Data":"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738"} Jul 13 20:28:34.865427 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:34.865330 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerStarted","Data":"88d78d69568e08bd9625b6237b9608c01d3d37b5cc66bbe5d21b14f98beb3b71"} Jul 13 20:28:35.054220 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.054196 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:28:35.056386 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:28:35.056359 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb60cc8a4_c2c4_46f5_87f9_073f3a8eee20.slice/crio-ab49106527c8fd2c2cdbaf01f141d930b10295b4ff675a77ec802b356d183b4b WatchSource:0}: Error finding container ab49106527c8fd2c2cdbaf01f141d930b10295b4ff675a77ec802b356d183b4b: Status 404 returned error can't find the container with id ab49106527c8fd2c2cdbaf01f141d930b10295b4ff675a77ec802b356d183b4b Jul 13 20:28:35.069683 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.069654 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:35.072550 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.072529 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") pod \"isvc-xgboost-graph-predictor-555fc69d56-xcfvp\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:35.281938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.281903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:28:35.440292 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.440257 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:28:35.443422 ip-10-0-128-52 kubenswrapper[2580]: W0713 20:28:35.443390 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98a6fde5_c82d_4a6d_a057_fdc3399a2025.slice/crio-12fb123e8b19ed8d91a922533cabdd67ca284ce3fc00aff1395764b26324392e WatchSource:0}: Error finding container 12fb123e8b19ed8d91a922533cabdd67ca284ce3fc00aff1395764b26324392e: Status 404 returned error can't find the container with id 12fb123e8b19ed8d91a922533cabdd67ca284ce3fc00aff1395764b26324392e Jul 13 20:28:35.871812 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.871777 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerStarted","Data":"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3"} Jul 13 20:28:35.872017 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.871845 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerStarted","Data":"12fb123e8b19ed8d91a922533cabdd67ca284ce3fc00aff1395764b26324392e"} Jul 13 20:28:35.873448 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.873418 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerStarted","Data":"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6"} Jul 13 20:28:35.873572 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:35.873479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerStarted","Data":"ab49106527c8fd2c2cdbaf01f141d930b10295b4ff675a77ec802b356d183b4b"} Jul 13 20:28:38.891634 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:38.891595 2580 generic.go:356] "Generic (PLEG): container finished" podID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerID="8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6" exitCode=0 Jul 13 20:28:38.892079 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:38.891669 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerDied","Data":"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6"} Jul 13 20:28:38.893362 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:38.893338 2580 generic.go:356] "Generic (PLEG): container finished" podID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerID="856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738" exitCode=0 Jul 13 20:28:38.893455 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:38.893363 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerDied","Data":"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738"} Jul 13 20:28:39.914134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:39.913251 2580 generic.go:356] "Generic (PLEG): container finished" podID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerID="3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3" exitCode=0 Jul 13 20:28:39.914134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:28:39.913307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerDied","Data":"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3"} Jul 13 20:29:08.122335 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:08.122291 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerStarted","Data":"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6"} Jul 13 20:29:08.126601 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:08.126495 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerStarted","Data":"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45"} Jul 13 20:29:08.128938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:08.128859 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerStarted","Data":"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f"} Jul 13 20:29:10.142838 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.142769 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerStarted","Data":"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9"} Jul 13 20:29:10.143219 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.142925 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:29:10.144834 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.144810 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerStarted","Data":"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b"} Jul 13 20:29:10.144950 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.144855 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:29:10.144950 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.144877 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:29:10.146837 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.146812 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:10.147222 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.147206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerStarted","Data":"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995"} Jul 13 20:29:10.147359 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.147345 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:29:10.147403 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.147383 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:29:10.148505 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.148484 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:10.167630 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.167589 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podStartSLOduration=2.602970683 podStartE2EDuration="36.16757611s" totalImagesPullingTime="29.991271942s" totalInitContainerRuntime="3.573333485s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:28:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:29:10.16634731 +0000 UTC m=+596.330906663" watchObservedRunningTime="2026-07-13 20:29:10.16757611 +0000 UTC m=+596.332135460" Jul 13 20:29:10.190546 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.190494 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podStartSLOduration=2.270244256 podStartE2EDuration="37.190477462s" totalImagesPullingTime="31.009695565s" totalInitContainerRuntime="3.910537641s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:28:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:29:10.188579464 +0000 UTC m=+596.353138816" watchObservedRunningTime="2026-07-13 20:29:10.190477462 +0000 UTC m=+596.355036814" Jul 13 20:29:10.217031 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:10.216986 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podStartSLOduration=1.664216272 podStartE2EDuration="36.216974332s" totalImagesPullingTime="30.965571744s" totalInitContainerRuntime="3.587186316s" isStatefulPod=false podCreationTimestamp="2026-07-13 20:28:34 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 20:29:10.214656718 +0000 UTC m=+596.379216069" watchObservedRunningTime="2026-07-13 20:29:10.216974332 +0000 UTC m=+596.381533683" Jul 13 20:29:11.152641 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:11.152598 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:11.153076 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:11.152599 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:11.153076 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:11.152950 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:29:11.154080 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:11.154060 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:12.156650 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:12.156603 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:15.801930 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:15.801903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:29:15.808188 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:15.801977 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:29:16.158642 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:16.158553 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:29:16.158836 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:16.158667 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:29:16.159188 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:16.159159 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:16.159276 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:16.159158 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:17.162084 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:17.162054 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:29:17.162610 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:17.162585 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:26.159548 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:26.159509 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:26.159548 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:26.159531 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:27.163037 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:27.162998 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:36.159884 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:36.159807 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:36.160402 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:36.159806 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:37.162795 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:37.162727 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:46.159108 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:46.159064 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:46.159624 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:46.159077 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:47.162541 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:47.162496 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:29:56.159648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:56.159615 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:29:56.160097 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:56.159610 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:29:57.162678 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:29:57.162639 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:30:06.159648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:06.159612 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:30:06.160107 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:06.159601 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:30:07.163913 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:07.163883 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:30:16.159134 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:16.159097 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:30:16.159596 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:16.159096 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:30:26.160665 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:26.160636 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:30:26.161266 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:30:26.160714 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:31:04.013715 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.013683 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:31:04.014226 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.013959 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" containerID="cri-o://da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f" gracePeriod=30 Jul 13 20:31:04.014226 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.013998 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kube-rbac-proxy" containerID="cri-o://33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9" gracePeriod=30 Jul 13 20:31:04.064583 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.064557 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:31:04.064839 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.064814 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" containerID="cri-o://8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6" gracePeriod=30 Jul 13 20:31:04.064917 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.064827 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kube-rbac-proxy" containerID="cri-o://bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b" gracePeriod=30 Jul 13 20:31:04.136222 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.136181 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:31:04.136505 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.136478 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" containerID="cri-o://2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45" gracePeriod=30 Jul 13 20:31:04.136566 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.136504 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kube-rbac-proxy" containerID="cri-o://da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995" gracePeriod=30 Jul 13 20:31:04.700626 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.700547 2580 generic.go:356] "Generic (PLEG): container finished" podID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerID="da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995" exitCode=2 Jul 13 20:31:04.700828 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.700620 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerDied","Data":"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995"} Jul 13 20:31:04.702869 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.702846 2580 generic.go:356] "Generic (PLEG): container finished" podID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerID="33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9" exitCode=2 Jul 13 20:31:04.702977 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.702926 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerDied","Data":"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9"} Jul 13 20:31:04.704929 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.704908 2580 generic.go:356] "Generic (PLEG): container finished" podID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerID="bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b" exitCode=2 Jul 13 20:31:04.705019 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:04.704965 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerDied","Data":"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b"} Jul 13 20:31:06.152841 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:06.152803 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.32:8643/healthz\": dial tcp 10.133.0.32:8643: connect: connection refused" Jul 13 20:31:06.153208 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:06.152808 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.30:8643/healthz\": dial tcp 10.133.0.30:8643: connect: connection refused" Jul 13 20:31:06.159218 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:06.159192 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.30:8080: connect: connection refused" Jul 13 20:31:06.159268 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:06.159195 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.32:8080: connect: connection refused" Jul 13 20:31:07.157100 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:07.157067 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kube-rbac-proxy" probeResult="failure" output="Get \"https://10.133.0.31:8643/healthz\": dial tcp 10.133.0.31:8643: connect: connection refused" Jul 13 20:31:07.162512 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:07.162490 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" probeResult="failure" output="dial tcp 10.133.0.31:8080: connect: connection refused" Jul 13 20:31:07.972881 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:07.972858 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:31:08.050481 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050459 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") pod \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " Jul 13 20:31:08.050648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050520 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config\") pod \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " Jul 13 20:31:08.050648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050545 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location\") pod \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " Jul 13 20:31:08.050648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050590 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert\") pod \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " Jul 13 20:31:08.050648 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050618 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9mvbv\" (UniqueName: \"kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv\") pod \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\" (UID: \"98a6fde5-c82d-4a6d-a057-fdc3399a2025\") " Jul 13 20:31:08.050961 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050930 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "98a6fde5-c82d-4a6d-a057-fdc3399a2025" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:31:08.051024 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050964 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "98a6fde5-c82d-4a6d-a057-fdc3399a2025" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:08.051024 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.050963 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "98a6fde5-c82d-4a6d-a057-fdc3399a2025" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:08.052916 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.052892 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv" (OuterVolumeSpecName: "kube-api-access-9mvbv") pod "98a6fde5-c82d-4a6d-a057-fdc3399a2025" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025"). InnerVolumeSpecName "kube-api-access-9mvbv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:31:08.052916 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.052905 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "98a6fde5-c82d-4a6d-a057-fdc3399a2025" (UID: "98a6fde5-c82d-4a6d-a057-fdc3399a2025"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:31:08.151435 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.151397 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:08.151435 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.151431 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kserve-provision-location\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:08.151435 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.151442 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/98a6fde5-c82d-4a6d-a057-fdc3399a2025-cabundle-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:08.151639 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.151452 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9mvbv\" (UniqueName: \"kubernetes.io/projected/98a6fde5-c82d-4a6d-a057-fdc3399a2025-kube-api-access-9mvbv\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:08.151639 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.151462 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/98a6fde5-c82d-4a6d-a057-fdc3399a2025-proxy-tls\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:08.731439 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.731350 2580 generic.go:356] "Generic (PLEG): container finished" podID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerID="da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f" exitCode=0 Jul 13 20:31:08.731439 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.731412 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" Jul 13 20:31:08.731439 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.731428 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerDied","Data":"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f"} Jul 13 20:31:08.731941 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.731488 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp" event={"ID":"98a6fde5-c82d-4a6d-a057-fdc3399a2025","Type":"ContainerDied","Data":"12fb123e8b19ed8d91a922533cabdd67ca284ce3fc00aff1395764b26324392e"} Jul 13 20:31:08.731941 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.731516 2580 scope.go:117] "RemoveContainer" containerID="33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9" Jul 13 20:31:08.739989 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.739963 2580 scope.go:117] "RemoveContainer" containerID="da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f" Jul 13 20:31:08.747491 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.747476 2580 scope.go:117] "RemoveContainer" containerID="3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3" Jul 13 20:31:08.751174 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.751151 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:31:08.755728 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.755713 2580 scope.go:117] "RemoveContainer" containerID="33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9" Jul 13 20:31:08.756097 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:08.756075 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9\": container with ID starting with 33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9 not found: ID does not exist" containerID="33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9" Jul 13 20:31:08.756178 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756108 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9"} err="failed to get container status \"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9\": rpc error: code = NotFound desc = could not find container \"33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9\": container with ID starting with 33bf2c4e2f6111a5768a49b0314c34f9f85ede7cea9cf7dc40b28a0951a282a9 not found: ID does not exist" Jul 13 20:31:08.756178 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756132 2580 scope.go:117] "RemoveContainer" containerID="da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f" Jul 13 20:31:08.756382 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:08.756368 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f\": container with ID starting with da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f not found: ID does not exist" containerID="da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f" Jul 13 20:31:08.756438 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756390 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f"} err="failed to get container status \"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f\": rpc error: code = NotFound desc = could not find container \"da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f\": container with ID starting with da96b08bdf3368f508b55d35f873291e0e9c04f7afe5456e841352f3a511217f not found: ID does not exist" Jul 13 20:31:08.756438 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756410 2580 scope.go:117] "RemoveContainer" containerID="3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3" Jul 13 20:31:08.756644 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:08.756628 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3\": container with ID starting with 3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3 not found: ID does not exist" containerID="3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3" Jul 13 20:31:08.756692 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756649 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3"} err="failed to get container status \"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3\": rpc error: code = NotFound desc = could not find container \"3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3\": container with ID starting with 3610070ba207cf711255f793a26ef09436e41ecda9d10698f48a3e2db2a543c3 not found: ID does not exist" Jul 13 20:31:08.756969 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:08.756953 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-xgboost-graph-predictor-555fc69d56-xcfvp"] Jul 13 20:31:09.341212 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.341142 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:31:09.343904 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.343887 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:31:09.464632 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464600 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location\") pod \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " Jul 13 20:31:09.464847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464640 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert\") pod \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " Jul 13 20:31:09.464847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464668 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config\") pod \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " Jul 13 20:31:09.464847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464695 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location\") pod \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " Jul 13 20:31:09.464847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464774 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls\") pod \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " Jul 13 20:31:09.464847 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464840 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f2jxg\" (UniqueName: \"kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg\") pod \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464878 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert\") pod \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464905 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config\") pod \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464932 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls\") pod \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\" (UID: \"811dd1e2-0d2a-4aab-8a91-be98ac0f7946\") " Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.464959 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-44d29\" (UniqueName: \"kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29\") pod \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\" (UID: \"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20\") " Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465028 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" (UID: "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465040 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location" (OuterVolumeSpecName: "kserve-provision-location") pod "811dd1e2-0d2a-4aab-8a91-be98ac0f7946" (UID: "811dd1e2-0d2a-4aab-8a91-be98ac0f7946"). InnerVolumeSpecName "kserve-provision-location". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465051 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" (UID: "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:09.465111 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465056 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "811dd1e2-0d2a-4aab-8a91-be98ac0f7946" (UID: "811dd1e2-0d2a-4aab-8a91-be98ac0f7946"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:09.465503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465339 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kserve-provision-location\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.465503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465360 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-cabundle-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.465503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465375 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.465503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465390 2580 reconciler_common.go:299] "Volume detached for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kserve-provision-location\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.465503 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465420 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config" (OuterVolumeSpecName: "kube-rbac-proxy-sar-config") pod "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" (UID: "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20"). InnerVolumeSpecName "kube-rbac-proxy-sar-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:09.465933 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.465526 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert" (OuterVolumeSpecName: "cabundle-cert") pod "811dd1e2-0d2a-4aab-8a91-be98ac0f7946" (UID: "811dd1e2-0d2a-4aab-8a91-be98ac0f7946"). InnerVolumeSpecName "cabundle-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 13 20:31:09.467139 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.467106 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" (UID: "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:31:09.467261 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.467230 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29" (OuterVolumeSpecName: "kube-api-access-44d29") pod "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" (UID: "b60cc8a4-c2c4-46f5-87f9-073f3a8eee20"). InnerVolumeSpecName "kube-api-access-44d29". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:31:09.467307 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.467266 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "811dd1e2-0d2a-4aab-8a91-be98ac0f7946" (UID: "811dd1e2-0d2a-4aab-8a91-be98ac0f7946"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 13 20:31:09.467307 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.467272 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg" (OuterVolumeSpecName: "kube-api-access-f2jxg") pod "811dd1e2-0d2a-4aab-8a91-be98ac0f7946" (UID: "811dd1e2-0d2a-4aab-8a91-be98ac0f7946"). InnerVolumeSpecName "kube-api-access-f2jxg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 13 20:31:09.566365 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566326 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-proxy-tls\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.566365 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566357 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f2jxg\" (UniqueName: \"kubernetes.io/projected/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-kube-api-access-f2jxg\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.566365 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566367 2580 reconciler_common.go:299] "Volume detached for volume \"cabundle-cert\" (UniqueName: \"kubernetes.io/configmap/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-cabundle-cert\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.566583 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566376 2580 reconciler_common.go:299] "Volume detached for volume \"kube-rbac-proxy-sar-config\" (UniqueName: \"kubernetes.io/configmap/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-rbac-proxy-sar-config\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.566583 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566384 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/811dd1e2-0d2a-4aab-8a91-be98ac0f7946-proxy-tls\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.566583 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.566393 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-44d29\" (UniqueName: \"kubernetes.io/projected/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20-kube-api-access-44d29\") on node \"ip-10-0-128-52.ec2.internal\" DevicePath \"\"" Jul 13 20:31:09.738504 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.738406 2580 generic.go:356] "Generic (PLEG): container finished" podID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerID="8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6" exitCode=0 Jul 13 20:31:09.738504 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.738466 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" Jul 13 20:31:09.738504 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.738478 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerDied","Data":"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6"} Jul 13 20:31:09.738986 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.738543 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv" event={"ID":"b60cc8a4-c2c4-46f5-87f9-073f3a8eee20","Type":"ContainerDied","Data":"ab49106527c8fd2c2cdbaf01f141d930b10295b4ff675a77ec802b356d183b4b"} Jul 13 20:31:09.738986 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.738581 2580 scope.go:117] "RemoveContainer" containerID="bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b" Jul 13 20:31:09.740544 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.740518 2580 generic.go:356] "Generic (PLEG): container finished" podID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerID="2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45" exitCode=0 Jul 13 20:31:09.740662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.740563 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" Jul 13 20:31:09.740662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.740590 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerDied","Data":"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45"} Jul 13 20:31:09.740662 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.740656 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd" event={"ID":"811dd1e2-0d2a-4aab-8a91-be98ac0f7946","Type":"ContainerDied","Data":"88d78d69568e08bd9625b6237b9608c01d3d37b5cc66bbe5d21b14f98beb3b71"} Jul 13 20:31:09.748938 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.748915 2580 scope.go:117] "RemoveContainer" containerID="8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6" Jul 13 20:31:09.756633 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.756619 2580 scope.go:117] "RemoveContainer" containerID="8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6" Jul 13 20:31:09.764532 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.764513 2580 scope.go:117] "RemoveContainer" containerID="bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b" Jul 13 20:31:09.764868 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.764850 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b\": container with ID starting with bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b not found: ID does not exist" containerID="bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b" Jul 13 20:31:09.764936 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.764878 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b"} err="failed to get container status \"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b\": rpc error: code = NotFound desc = could not find container \"bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b\": container with ID starting with bfb85c26ef4ad82d5f79d275f4e457622b81f7f19ad9190d4616267f8631dc6b not found: ID does not exist" Jul 13 20:31:09.764936 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.764895 2580 scope.go:117] "RemoveContainer" containerID="8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6" Jul 13 20:31:09.765171 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.765139 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6\": container with ID starting with 8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6 not found: ID does not exist" containerID="8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6" Jul 13 20:31:09.765220 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.765180 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6"} err="failed to get container status \"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6\": rpc error: code = NotFound desc = could not find container \"8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6\": container with ID starting with 8e502a8f3bdb71cf04f3702fdeb76a8a3aa4714795cfb6b1fcff87f34894d9f6 not found: ID does not exist" Jul 13 20:31:09.765220 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.765202 2580 scope.go:117] "RemoveContainer" containerID="8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6" Jul 13 20:31:09.765425 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.765408 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6\": container with ID starting with 8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6 not found: ID does not exist" containerID="8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6" Jul 13 20:31:09.765461 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.765427 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:31:09.765461 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.765432 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6"} err="failed to get container status \"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6\": rpc error: code = NotFound desc = could not find container \"8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6\": container with ID starting with 8e4e19b19d6fac4349bea3958b916e5db4ecce1a91ced15c01e20eec3ec920c6 not found: ID does not exist" Jul 13 20:31:09.765461 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.765449 2580 scope.go:117] "RemoveContainer" containerID="da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995" Jul 13 20:31:09.770004 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.769984 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-1-predictor-6449ff9c59-zvxqd"] Jul 13 20:31:09.773239 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.773212 2580 scope.go:117] "RemoveContainer" containerID="2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45" Jul 13 20:31:09.781069 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.781052 2580 scope.go:117] "RemoveContainer" containerID="856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738" Jul 13 20:31:09.781256 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.781235 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:31:09.789430 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.789324 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/isvc-sklearn-graph-2-predictor-7894f545fc-xvkwv"] Jul 13 20:31:09.789484 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.789455 2580 scope.go:117] "RemoveContainer" containerID="da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995" Jul 13 20:31:09.789703 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.789684 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995\": container with ID starting with da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995 not found: ID does not exist" containerID="da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995" Jul 13 20:31:09.789793 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.789713 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995"} err="failed to get container status \"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995\": rpc error: code = NotFound desc = could not find container \"da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995\": container with ID starting with da7ca8e6ca794a893fbbbf2705bbdf94012db8c10471f4438bd15378f5f44995 not found: ID does not exist" Jul 13 20:31:09.789793 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.789763 2580 scope.go:117] "RemoveContainer" containerID="2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45" Jul 13 20:31:09.790026 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.790007 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45\": container with ID starting with 2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45 not found: ID does not exist" containerID="2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45" Jul 13 20:31:09.790068 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.790034 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45"} err="failed to get container status \"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45\": rpc error: code = NotFound desc = could not find container \"2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45\": container with ID starting with 2b82099c0796be9790a1e0fc7aa33610189cf9d30f34a5e92fe7680f94423b45 not found: ID does not exist" Jul 13 20:31:09.790068 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.790049 2580 scope.go:117] "RemoveContainer" containerID="856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738" Jul 13 20:31:09.790263 ip-10-0-128-52 kubenswrapper[2580]: E0713 20:31:09.790245 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738\": container with ID starting with 856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738 not found: ID does not exist" containerID="856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738" Jul 13 20:31:09.790304 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:09.790268 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738"} err="failed to get container status \"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738\": rpc error: code = NotFound desc = could not find container \"856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738\": container with ID starting with 856317542ef25f91fe975bb328e91bfe51360015c63051edfc4cc34a0cffb738 not found: ID does not exist" Jul 13 20:31:10.341077 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:10.341043 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" path="/var/lib/kubelet/pods/811dd1e2-0d2a-4aab-8a91-be98ac0f7946/volumes" Jul 13 20:31:10.341536 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:10.341522 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" path="/var/lib/kubelet/pods/98a6fde5-c82d-4a6d-a057-fdc3399a2025/volumes" Jul 13 20:31:10.342265 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:31:10.342250 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" path="/var/lib/kubelet/pods/b60cc8a4-c2c4-46f5-87f9-073f3a8eee20/volumes" Jul 13 20:34:15.830253 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:34:15.830141 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:34:15.834365 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:34:15.832029 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:39:15.857086 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:39:15.856978 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:39:15.861065 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:39:15.859644 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:44:15.886892 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:44:15.886782 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:44:15.890893 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:44:15.889024 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:49:15.920410 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:49:15.920305 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:49:15.926154 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:49:15.923652 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:54:15.947505 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:54:15.947397 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:54:15.951779 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:54:15.951764 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:59:15.975045 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:59:15.974941 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 20:59:15.981541 ip-10-0-128-52 kubenswrapper[2580]: I0713 20:59:15.981523 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 21:04:16.002913 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:04:16.002796 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 21:04:16.009025 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:04:16.009007 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 21:08:49.107479 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:49.107396 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-p2ps2_0bd6ef49-c50c-44bc-b547-95958e3de5bb/global-pull-secret-syncer/0.log" Jul 13 21:08:49.234987 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:49.234958 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-w58l4_fb761904-bc7d-42fa-bc30-b4b5dfbc188e/konnectivity-agent/0.log" Jul 13 21:08:49.275148 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:49.275127 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-52.ec2.internal_a7eb5df1a0ecc6549b309baa4fc43939/haproxy/0.log" Jul 13 21:08:53.077086 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.077044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/alertmanager/0.log" Jul 13 21:08:53.104702 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.104679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/config-reloader/0.log" Jul 13 21:08:53.136019 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.135996 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/kube-rbac-proxy-web/0.log" Jul 13 21:08:53.163373 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.163352 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/kube-rbac-proxy/0.log" Jul 13 21:08:53.193085 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.193065 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/kube-rbac-proxy-metric/0.log" Jul 13 21:08:53.217907 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.217881 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/prom-label-proxy/0.log" Jul 13 21:08:53.244614 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.244596 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_943e3fe8-5e85-44f4-b987-77f8f7f30929/init-config-reloader/0.log" Jul 13 21:08:53.322142 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.322115 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlfjp_91168a65-5bd2-4517-82e0-9e3b7174cd31/kube-state-metrics/0.log" Jul 13 21:08:53.344464 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.344407 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlfjp_91168a65-5bd2-4517-82e0-9e3b7174cd31/kube-rbac-proxy-main/0.log" Jul 13 21:08:53.364558 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.364540 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-dlfjp_91168a65-5bd2-4517-82e0-9e3b7174cd31/kube-rbac-proxy-self/0.log" Jul 13 21:08:53.480992 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.480964 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4d5g_e6bb5201-1df2-4b42-84ce-25749bfc8e8f/node-exporter/0.log" Jul 13 21:08:53.507045 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.507014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4d5g_e6bb5201-1df2-4b42-84ce-25749bfc8e8f/kube-rbac-proxy/0.log" Jul 13 21:08:53.529932 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.529911 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-b4d5g_e6bb5201-1df2-4b42-84ce-25749bfc8e8f/init-textfile/0.log" Jul 13 21:08:53.725503 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.725416 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9cd2q_97efcf0a-cf78-411f-8548-2efc4d7e3563/kube-rbac-proxy-main/0.log" Jul 13 21:08:53.746079 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.746050 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9cd2q_97efcf0a-cf78-411f-8548-2efc4d7e3563/kube-rbac-proxy-self/0.log" Jul 13 21:08:53.774766 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:53.774710 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-9cd2q_97efcf0a-cf78-411f-8548-2efc4d7e3563/openshift-state-metrics/0.log" Jul 13 21:08:54.004852 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.004769 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-qrk7d_4def49c7-958f-4147-8a0d-9a5cb255c72b/prometheus-operator/0.log" Jul 13 21:08:54.028626 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.028596 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-qrk7d_4def49c7-958f-4147-8a0d-9a5cb255c72b/kube-rbac-proxy/0.log" Jul 13 21:08:54.065564 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.065527 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-6jjtz_28430f79-afd3-46ad-8693-90644bf594c2/prometheus-operator-admission-webhook/0.log" Jul 13 21:08:54.106968 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.106947 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cf956d57-fd8ht_eb095fcf-ff96-4854-9270-ee06adc81cb9/telemeter-client/0.log" Jul 13 21:08:54.140386 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.140360 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cf956d57-fd8ht_eb095fcf-ff96-4854-9270-ee06adc81cb9/reload/0.log" Jul 13 21:08:54.167374 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.167342 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-55cf956d57-fd8ht_eb095fcf-ff96-4854-9270-ee06adc81cb9/kube-rbac-proxy/0.log" Jul 13 21:08:54.205199 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.205159 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/thanos-query/0.log" Jul 13 21:08:54.240845 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.240807 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/kube-rbac-proxy-web/0.log" Jul 13 21:08:54.266646 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.266582 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/kube-rbac-proxy/0.log" Jul 13 21:08:54.294716 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.294699 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/prom-label-proxy/0.log" Jul 13 21:08:54.326173 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.326153 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/kube-rbac-proxy-rules/0.log" Jul 13 21:08:54.364955 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:54.364935 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-7d76897594-gcznm_635d7e74-d757-47ef-acd3-ff527b8a17d9/kube-rbac-proxy-metrics/0.log" Jul 13 21:08:56.603208 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:56.603182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74857fddc6-lpbtg_f4252770-8cfb-4f47-91e8-30e36230b268/console/0.log" Jul 13 21:08:56.634129 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:56.634096 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-2jvd6_4211d7ed-40ee-437f-be9a-62d9a13694fb/download-server/0.log" Jul 13 21:08:57.393566 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393533 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh"] Jul 13 21:08:57.393883 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393870 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="storage-initializer" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393885 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="storage-initializer" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393893 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="storage-initializer" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393898 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="storage-initializer" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393908 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393915 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393930 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393935 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" Jul 13 21:08:57.393941 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393942 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="storage-initializer" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393947 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="storage-initializer" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393954 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393959 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393966 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393971 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393977 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393982 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393992 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.393998 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394046 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kserve-container" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394053 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kserve-container" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394059 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b60cc8a4-c2c4-46f5-87f9-073f3a8eee20" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394066 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="98a6fde5-c82d-4a6d-a057-fdc3399a2025" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394072 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kube-rbac-proxy" Jul 13 21:08:57.394172 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.394078 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="811dd1e2-0d2a-4aab-8a91-be98ac0f7946" containerName="kserve-container" Jul 13 21:08:57.396924 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.396908 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.399125 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.399092 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xlsxm\"/\"kube-root-ca.crt\"" Jul 13 21:08:57.399642 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.399622 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xlsxm\"/\"openshift-service-ca.crt\"" Jul 13 21:08:57.399763 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.399636 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xlsxm\"/\"default-dockercfg-nr2xr\"" Jul 13 21:08:57.403663 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.403642 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh"] Jul 13 21:08:57.492016 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.491985 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kk2nx\" (UniqueName: \"kubernetes.io/projected/8ca05857-6687-4bd6-aab9-b7f37e92354a-kube-api-access-kk2nx\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.492170 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.492029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-podres\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.492170 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.492054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-sys\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.492170 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.492068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-proc\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.492312 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.492184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-lib-modules\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593175 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kk2nx\" (UniqueName: \"kubernetes.io/projected/8ca05857-6687-4bd6-aab9-b7f37e92354a-kube-api-access-kk2nx\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593326 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-podres\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593326 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593293 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-podres\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593326 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-sys\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593531 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-proc\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593531 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593403 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-sys\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593531 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-lib-modules\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593531 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593468 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-proc\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.593666 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.593568 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/8ca05857-6687-4bd6-aab9-b7f37e92354a-lib-modules\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.600139 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.600110 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kk2nx\" (UniqueName: \"kubernetes.io/projected/8ca05857-6687-4bd6-aab9-b7f37e92354a-kube-api-access-kk2nx\") pod \"perf-node-gather-daemonset-pq5nh\" (UID: \"8ca05857-6687-4bd6-aab9-b7f37e92354a\") " pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.707658 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.707591 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:57.835064 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:57.835035 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh"] Jul 13 21:08:57.837620 ip-10-0-128-52 kubenswrapper[2580]: W0713 21:08:57.837585 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod8ca05857_6687_4bd6_aab9_b7f37e92354a.slice/crio-839f07770d2e8eed8df9b11d26b752b335345ef30102dfd374550a47888cd75d WatchSource:0}: Error finding container 839f07770d2e8eed8df9b11d26b752b335345ef30102dfd374550a47888cd75d: Status 404 returned error can't find the container with id 839f07770d2e8eed8df9b11d26b752b335345ef30102dfd374550a47888cd75d Jul 13 21:08:58.066047 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.066021 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrhss_911cd773-52de-4103-9df0-21fd8acbc674/dns/0.log" Jul 13 21:08:58.087551 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.087526 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" event={"ID":"8ca05857-6687-4bd6-aab9-b7f37e92354a","Type":"ContainerStarted","Data":"1b15416ab9df46e8e7f2b2f7b5e3c3289521092e3661e723c6967098b2eacfee"} Jul 13 21:08:58.087693 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.087599 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" event={"ID":"8ca05857-6687-4bd6-aab9-b7f37e92354a","Type":"ContainerStarted","Data":"839f07770d2e8eed8df9b11d26b752b335345ef30102dfd374550a47888cd75d"} Jul 13 21:08:58.087693 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.087663 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:08:58.088477 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.088461 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-hrhss_911cd773-52de-4103-9df0-21fd8acbc674/kube-rbac-proxy/0.log" Jul 13 21:08:58.106196 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.106156 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" podStartSLOduration=1.106143295 podStartE2EDuration="1.106143295s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-13 21:08:57 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-13 21:08:58.103856186 +0000 UTC m=+2984.268415536" watchObservedRunningTime="2026-07-13 21:08:58.106143295 +0000 UTC m=+2984.270702646" Jul 13 21:08:58.166256 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.166230 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-69s4w_9853f083-cbfb-439a-8698-4f85d7007f17/dns-node-resolver/0.log" Jul 13 21:08:58.778119 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.778087 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-84fb4758cb-6fdhb_d6f94186-5457-40c0-abec-3331eb1ba492/registry/0.log" Jul 13 21:08:58.839347 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:58.839323 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-9wtbh_c1146270-62c3-4f1e-b917-b3c45c509559/node-ca/0.log" Jul 13 21:08:59.962126 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:08:59.962088 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-4c2pr_d83a99b0-2e4c-4d72-bea4-97d78b414c7e/serve-healthcheck-canary/0.log" Jul 13 21:09:00.397955 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:00.397929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p5kz_072e919e-d0f4-4058-a026-414ec546d911/kube-rbac-proxy/0.log" Jul 13 21:09:00.422693 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:00.422667 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p5kz_072e919e-d0f4-4058-a026-414ec546d911/exporter/0.log" Jul 13 21:09:00.446035 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:00.446014 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-7p5kz_072e919e-d0f4-4058-a026-414ec546d911/extractor/0.log" Jul 13 21:09:02.815973 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:02.815942 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-84fc959947-cn5tg_d7770607-d7c8-42a8-a416-1e63474c7fcd/manager/0.log" Jul 13 21:09:03.048165 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:03.048139 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-4qp95_17e5cb49-23e6-48b8-99b3-96a815e59d38/s3-init/0.log" Jul 13 21:09:03.100490 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:03.100418 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_s3-init-4qp95_17e5cb49-23e6-48b8-99b3-96a815e59d38/download-hf-model/0.log" Jul 13 21:09:04.102352 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:04.102323 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xlsxm/perf-node-gather-daemonset-pq5nh" Jul 13 21:09:08.616709 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.616679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/kube-multus-additional-cni-plugins/0.log" Jul 13 21:09:08.637667 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.637642 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/egress-router-binary-copy/0.log" Jul 13 21:09:08.660460 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.660443 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/cni-plugins/0.log" Jul 13 21:09:08.678293 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.678278 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/bond-cni-plugin/0.log" Jul 13 21:09:08.698421 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.698406 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/routeoverride-cni/0.log" Jul 13 21:09:08.718818 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.718802 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/whereabouts-cni-bincopy/0.log" Jul 13 21:09:08.737307 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.737280 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-tw92n_81d105e4-a5b9-426c-988d-3988cf6b3706/whereabouts-cni/0.log" Jul 13 21:09:08.833123 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.833104 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-zk9nn_2962d625-be0c-4c1e-ae90-f2577f6e5f20/kube-multus/0.log" Jul 13 21:09:08.942424 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.942394 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-shvr5_58ec379f-96e0-41d3-a09e-bdc06bd97c08/network-metrics-daemon/0.log" Jul 13 21:09:08.961585 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:08.961568 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-shvr5_58ec379f-96e0-41d3-a09e-bdc06bd97c08/kube-rbac-proxy/0.log" Jul 13 21:09:09.656727 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.656699 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-controller/0.log" Jul 13 21:09:09.673811 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.673788 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/0.log" Jul 13 21:09:09.673925 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.673834 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovn-acl-logging/1.log" Jul 13 21:09:09.691327 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.691303 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/kube-rbac-proxy-node/0.log" Jul 13 21:09:09.713889 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.713866 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/kube-rbac-proxy-ovn-metrics/0.log" Jul 13 21:09:09.733363 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.733343 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/northd/0.log" Jul 13 21:09:09.752412 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.752391 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/nbdb/0.log" Jul 13 21:09:09.770921 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.770905 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/sbdb/0.log" Jul 13 21:09:09.813574 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:09.813557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-4sksc_f63aee13-3d39-4eeb-b7c5-eecd7fe21d0f/ovnkube-controller/0.log" Jul 13 21:09:11.249249 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:11.249221 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-ds8ss_58065293-dd9f-4398-8e33-f48831027fda/network-check-target-container/0.log" Jul 13 21:09:12.158474 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:12.158446 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-7ccwn_d0626783-392d-4d89-8c21-0e1f585e7fb5/iptables-alerter/0.log" Jul 13 21:09:12.866699 ip-10-0-128-52 kubenswrapper[2580]: I0713 21:09:12.866669 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-l25vp_b20e3596-ac0d-4fa6-9c6c-3b4eb6a49b90/tuned/0.log"