Jul 02 16:54:31.923405 ip-10-0-130-120 systemd[1]: Starting Kubernetes Kubelet... Jul 02 16:54:32.317745 ip-10-0-130-120 kubenswrapper[2565]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:32.317745 ip-10-0-130-120 kubenswrapper[2565]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 02 16:54:32.317745 ip-10-0-130-120 kubenswrapper[2565]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:32.317745 ip-10-0-130-120 kubenswrapper[2565]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 02 16:54:32.317745 ip-10-0-130-120 kubenswrapper[2565]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 02 16:54:32.318424 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.317825 2565 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 02 16:54:32.323159 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323144 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.323159 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323159 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323163 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323166 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323169 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323172 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323175 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323177 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323180 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323182 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323186 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323189 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323191 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323194 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323197 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323200 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323202 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323205 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323207 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323210 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323213 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.323237 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323216 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323218 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323221 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323223 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323226 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323228 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323231 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323233 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323236 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323238 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323241 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323244 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323247 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323250 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323253 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323255 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323260 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323264 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323267 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.323716 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323270 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323272 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323275 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323278 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323280 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323282 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323285 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323288 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323292 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323296 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323298 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323301 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323303 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323305 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323309 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323311 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323314 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323317 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323320 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.324226 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323322 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323325 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323327 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323330 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323332 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323335 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323337 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323339 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323342 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323344 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323346 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323349 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323353 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323355 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323358 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323361 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323363 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323365 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323368 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323372 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.324683 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323375 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323377 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323380 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323383 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323385 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323388 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323391 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323393 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323396 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323399 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323401 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323404 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323406 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323409 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323411 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323414 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323416 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323419 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323421 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.323423 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.325196 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324130 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324137 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324139 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324142 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324145 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324153 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324156 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324159 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324162 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324164 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324169 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324173 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324176 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324179 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324183 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324187 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324190 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324194 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324197 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.325679 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324201 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324204 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324207 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324210 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324213 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324215 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324218 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324220 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324223 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324225 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324228 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324230 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324233 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324236 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324238 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324241 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324243 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324246 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324248 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.326162 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324256 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324259 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324261 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324264 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324267 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324270 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324272 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324274 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324277 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324279 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324282 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324284 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324286 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324289 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324291 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324294 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324297 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324299 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324301 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324303 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.326638 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324306 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324308 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324311 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324313 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324315 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324317 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324320 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324322 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324325 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324328 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324330 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324333 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324335 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324343 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324349 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324351 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324354 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324362 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324365 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324368 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.327144 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324370 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324372 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324375 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324377 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324379 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324382 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324384 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324387 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324390 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324392 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324394 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324397 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324399 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324402 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324404 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324407 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324409 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324411 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324414 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324416 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.327721 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324419 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324791 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324796 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324798 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324801 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324804 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324806 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324808 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324811 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324813 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324816 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324819 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324821 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324823 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324826 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324830 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324832 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324835 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324838 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.328254 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324841 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324843 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324846 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324848 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324851 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324853 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324856 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324858 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324860 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324863 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324865 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324867 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324870 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324873 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324875 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324878 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324880 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324882 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324885 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324888 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.328702 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324890 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324893 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324895 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324897 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324900 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324902 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324905 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324907 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324909 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324912 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324914 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324919 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324922 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324924 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324927 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324929 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324932 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324934 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324937 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324941 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.329208 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324945 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324948 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324951 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324953 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324956 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324959 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324962 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324964 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324967 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324971 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324974 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324976 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324979 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324981 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324984 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324986 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324989 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324991 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324994 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324996 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.329727 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.324998 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325001 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325003 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325007 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325010 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325013 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325015 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325018 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325020 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325023 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325025 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325027 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325030 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325032 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325035 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325038 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325040 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325043 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325045 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325048 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.330231 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325050 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325134 2565 flags.go:64] FLAG: --address="0.0.0.0" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325141 2565 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325146 2565 flags.go:64] FLAG: --anonymous-auth="true" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325151 2565 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325155 2565 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325159 2565 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325163 2565 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325166 2565 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325170 2565 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325172 2565 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325176 2565 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325179 2565 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325183 2565 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325186 2565 flags.go:64] FLAG: --cgroup-root="" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325190 2565 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325193 2565 flags.go:64] FLAG: --client-ca-file="" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325196 2565 flags.go:64] FLAG: --cloud-provider="external" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325199 2565 flags.go:64] FLAG: --cluster-dns="[]" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325203 2565 flags.go:64] FLAG: --cluster-domain="" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325206 2565 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325209 2565 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325212 2565 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325215 2565 flags.go:64] FLAG: --container-log-max-files="5" Jul 02 16:54:32.330718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325219 2565 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325221 2565 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325224 2565 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325228 2565 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325230 2565 flags.go:64] FLAG: --contention-profiling="false" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325233 2565 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325236 2565 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325239 2565 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325242 2565 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325249 2565 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325251 2565 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325254 2565 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325256 2565 flags.go:64] FLAG: --enable-load-reader="false" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325259 2565 flags.go:64] FLAG: --enable-server="true" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325263 2565 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325267 2565 flags.go:64] FLAG: --event-burst="100" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325270 2565 flags.go:64] FLAG: --event-qps="50" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325272 2565 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325275 2565 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325278 2565 flags.go:64] FLAG: --eviction-hard="" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325281 2565 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325287 2565 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325289 2565 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325294 2565 flags.go:64] FLAG: --eviction-soft="" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325296 2565 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 02 16:54:32.331346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325299 2565 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325302 2565 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325305 2565 flags.go:64] FLAG: --experimental-mounter-path="" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325307 2565 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325310 2565 flags.go:64] FLAG: --fail-swap-on="true" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325313 2565 flags.go:64] FLAG: --feature-gates="" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325316 2565 flags.go:64] FLAG: --file-check-frequency="20s" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325319 2565 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325322 2565 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325325 2565 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325328 2565 flags.go:64] FLAG: --healthz-port="10248" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325331 2565 flags.go:64] FLAG: --help="false" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325334 2565 flags.go:64] FLAG: --hostname-override="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325337 2565 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325340 2565 flags.go:64] FLAG: --http-check-frequency="20s" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325342 2565 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325345 2565 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325348 2565 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325351 2565 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325354 2565 flags.go:64] FLAG: --image-service-endpoint="" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325357 2565 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325359 2565 flags.go:64] FLAG: --kube-api-burst="100" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325362 2565 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325366 2565 flags.go:64] FLAG: --kube-api-qps="50" Jul 02 16:54:32.331996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325368 2565 flags.go:64] FLAG: --kube-reserved="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325371 2565 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325374 2565 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325377 2565 flags.go:64] FLAG: --kubelet-cgroups="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325381 2565 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325383 2565 flags.go:64] FLAG: --lock-file="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325387 2565 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325390 2565 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325393 2565 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325398 2565 flags.go:64] FLAG: --log-json-split-stream="false" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325400 2565 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325403 2565 flags.go:64] FLAG: --log-text-split-stream="false" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325406 2565 flags.go:64] FLAG: --logging-format="text" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325408 2565 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325412 2565 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325415 2565 flags.go:64] FLAG: --manifest-url="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325418 2565 flags.go:64] FLAG: --manifest-url-header="" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325422 2565 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325424 2565 flags.go:64] FLAG: --max-open-files="1000000" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325428 2565 flags.go:64] FLAG: --max-pods="110" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325431 2565 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325434 2565 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325437 2565 flags.go:64] FLAG: --memory-manager-policy="None" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325439 2565 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325442 2565 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 02 16:54:32.332607 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325445 2565 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325447 2565 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325455 2565 flags.go:64] FLAG: --node-status-max-images="50" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325458 2565 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325460 2565 flags.go:64] FLAG: --oom-score-adj="-999" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325463 2565 flags.go:64] FLAG: --pod-cidr="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325466 2565 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325472 2565 flags.go:64] FLAG: --pod-manifest-path="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325475 2565 flags.go:64] FLAG: --pod-max-pids="-1" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325478 2565 flags.go:64] FLAG: --pods-per-core="0" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325480 2565 flags.go:64] FLAG: --port="10250" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325484 2565 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325487 2565 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-03d673ba22d882ec7" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325490 2565 flags.go:64] FLAG: --qos-reserved="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325494 2565 flags.go:64] FLAG: --read-only-port="10255" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325497 2565 flags.go:64] FLAG: --register-node="true" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325499 2565 flags.go:64] FLAG: --register-with-taints="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325503 2565 flags.go:64] FLAG: --registry-burst="10" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325505 2565 flags.go:64] FLAG: --registry-qps="5" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325508 2565 flags.go:64] FLAG: --reserved-cpus="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325511 2565 flags.go:64] FLAG: --reserved-memory="" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325514 2565 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325517 2565 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325520 2565 flags.go:64] FLAG: --rotate-certificates="false" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325522 2565 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 02 16:54:32.333246 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325525 2565 flags.go:64] FLAG: --runonce="false" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325528 2565 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325531 2565 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325533 2565 flags.go:64] FLAG: --seccomp-default="false" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325536 2565 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325538 2565 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325541 2565 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325544 2565 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325547 2565 flags.go:64] FLAG: --storage-driver-password="root" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325550 2565 flags.go:64] FLAG: --storage-driver-secure="false" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325553 2565 flags.go:64] FLAG: --storage-driver-table="stats" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325555 2565 flags.go:64] FLAG: --storage-driver-user="root" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325558 2565 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325560 2565 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325564 2565 flags.go:64] FLAG: --system-cgroups="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325566 2565 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325571 2565 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325573 2565 flags.go:64] FLAG: --tls-cert-file="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325577 2565 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325582 2565 flags.go:64] FLAG: --tls-min-version="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325584 2565 flags.go:64] FLAG: --tls-private-key-file="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325588 2565 flags.go:64] FLAG: --topology-manager-policy="none" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325591 2565 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325593 2565 flags.go:64] FLAG: --topology-manager-scope="container" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325596 2565 flags.go:64] FLAG: --v="2" Jul 02 16:54:32.333851 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325600 2565 flags.go:64] FLAG: --version="false" Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325603 2565 flags.go:64] FLAG: --vmodule="" Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325607 2565 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325610 2565 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325725 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325728 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325731 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325734 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325737 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325740 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325744 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325747 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325750 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325753 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325755 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325758 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325760 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325763 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325765 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325768 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.334462 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325770 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325773 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325776 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325778 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325781 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325784 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325788 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325791 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325795 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325797 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325800 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325803 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325805 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325808 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325810 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325813 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325815 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325818 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325820 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325823 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.334970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325825 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325828 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325830 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325832 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325835 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325837 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325840 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325842 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325845 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325847 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325849 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325852 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325854 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325857 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325859 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325862 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325865 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325868 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325870 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325873 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.335478 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325876 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325879 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325881 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325884 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325886 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325889 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325891 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325894 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325896 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325898 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325901 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325903 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325906 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325908 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325910 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325913 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325916 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325920 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325922 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.335965 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325924 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325927 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325929 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325932 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325934 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325936 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325940 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325942 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325945 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325948 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325951 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325954 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325956 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325961 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325963 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325966 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325968 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325971 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325973 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325975 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.336493 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325978 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325980 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325982 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.325985 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.325993 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333087 2565 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333101 2565 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333137 2565 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333144 2565 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333201 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333206 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333209 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333212 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333215 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333218 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.336970 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333221 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333223 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333226 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333228 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333231 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333234 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333236 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333238 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333241 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333244 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333246 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333249 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333252 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333255 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333257 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333260 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333263 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333265 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333268 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333271 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.337352 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333273 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333276 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333278 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333283 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333286 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333288 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333290 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333293 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333296 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333298 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333301 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333303 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333305 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333308 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333310 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333312 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333315 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333318 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333320 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333322 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.337869 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333325 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333329 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333333 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333336 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333340 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333343 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333346 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333349 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333351 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333354 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333357 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333359 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333362 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333365 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333368 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333372 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333376 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333378 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333381 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.338402 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333384 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333386 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333389 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333391 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333394 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333396 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333399 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333401 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333404 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333406 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333409 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333411 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333414 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333416 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333419 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333421 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333424 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333426 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333428 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333431 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.338864 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333433 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333435 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333438 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333440 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333443 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333445 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333448 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333450 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333453 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333456 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333459 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333462 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333464 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333467 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333472 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:32.339363 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333581 2565 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333586 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333589 2565 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333593 2565 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333597 2565 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333600 2565 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333603 2565 feature_gate.go:328] unrecognized feature gate: Example2 Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333605 2565 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333608 2565 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333611 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333614 2565 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333616 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333619 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333621 2565 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333623 2565 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333626 2565 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333628 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333631 2565 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333633 2565 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333636 2565 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 02 16:54:32.339728 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333638 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333640 2565 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333643 2565 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333645 2565 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333648 2565 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333650 2565 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333654 2565 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333656 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333659 2565 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333661 2565 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333663 2565 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333666 2565 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333668 2565 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333670 2565 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333673 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333675 2565 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333677 2565 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333680 2565 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333682 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333685 2565 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 02 16:54:32.340429 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333687 2565 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333689 2565 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333692 2565 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333694 2565 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333696 2565 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333699 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333701 2565 feature_gate.go:328] unrecognized feature gate: Example Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333703 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333706 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333708 2565 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333710 2565 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333713 2565 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333715 2565 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333717 2565 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333720 2565 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333722 2565 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333725 2565 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333727 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333730 2565 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333732 2565 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 02 16:54:32.341225 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333734 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333737 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333740 2565 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333742 2565 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333744 2565 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333746 2565 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333749 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333751 2565 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333754 2565 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333756 2565 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333758 2565 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333760 2565 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333763 2565 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333765 2565 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333767 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333770 2565 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333772 2565 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333774 2565 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333777 2565 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 02 16:54:32.341838 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333779 2565 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333782 2565 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333786 2565 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333789 2565 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333791 2565 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333794 2565 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333797 2565 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333800 2565 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333802 2565 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333805 2565 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333807 2565 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333810 2565 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333813 2565 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333816 2565 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333819 2565 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333822 2565 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333825 2565 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333827 2565 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333829 2565 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 02 16:54:32.342412 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:32.333832 2565 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333836 2565 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.333929 2565 server.go:962] "Client rotation is on, will bootstrap in background" Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.336142 2565 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.337542 2565 server.go:1019] "Starting client certificate rotation" Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.337647 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.337698 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 02 16:54:32.342883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.342020 2565 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 16:54:32.345266 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.345248 2565 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 02 16:54:32.353783 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.353768 2565 log.go:25] "Validated CRI v1 runtime API" Jul 02 16:54:32.357024 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.357008 2565 log.go:25] "Validated CRI v1 image API" Jul 02 16:54:32.367277 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.367260 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 16:54:32.369532 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.369513 2565 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 02 16:54:32.372154 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.372135 2565 fs.go:135] Filesystem UUIDs: map[6e45888e-160a-4808-a137-e18d8679bf47:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b8c641a3-654a-4ede-a0fe-ba73b8fd7a83:/dev/nvme0n1p4] Jul 02 16:54:32.372238 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.372153 2565 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 02 16:54:32.377929 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.377826 2565 manager.go:216] Machine: {Timestamp:2026-07-02 16:54:32.37654033 +0000 UTC m=+0.336889396 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3100842 MemoryCapacity:32812179456 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec29495341aca21bb03ad29a2b066f8c SystemUUID:ec294953-41ac-a21b-b03a-d29a2b066f8c BootID:08d11cc2-2ff4-4463-a2c7-ad8f319577cc Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406089728 Type:vfs Inodes:4005393 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562439168 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:a8:61:af:ef:33 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:a8:61:af:ef:33 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:7a:a7:5e:fe:dc:b0 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812179456 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 02 16:54:32.377929 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.377928 2565 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 02 16:54:32.378039 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.378026 2565 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 02 16:54:32.381155 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.381132 2565 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 02 16:54:32.381302 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.381157 2565 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-120.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 02 16:54:32.382596 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.382586 2565 topology_manager.go:138] "Creating topology manager with none policy" Jul 02 16:54:32.382630 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.382598 2565 container_manager_linux.go:309] "Creating device plugin manager" Jul 02 16:54:32.382630 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.382606 2565 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 16:54:32.383283 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.383273 2565 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 02 16:54:32.384390 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.384381 2565 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 02 16:54:32.388447 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388436 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 02 16:54:32.388564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388554 2565 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 02 16:54:32.388598 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388586 2565 kubelet.go:486] "Attempting to sync node with API server" Jul 02 16:54:32.388648 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388603 2565 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 02 16:54:32.388648 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388618 2565 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 02 16:54:32.388648 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388628 2565 kubelet.go:398] "Adding apiserver pod source" Jul 02 16:54:32.388648 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388636 2565 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 02 16:54:32.388979 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388967 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 16:54:32.389011 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.388985 2565 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 02 16:54:32.390973 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.390958 2565 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 02 16:54:32.393474 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.393459 2565 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 02 16:54:32.393548 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.393490 2565 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 02 16:54:32.393548 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.393494 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t68jx" Jul 02 16:54:32.394150 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394139 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 02 16:54:32.394188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394155 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 02 16:54:32.394188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394162 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 02 16:54:32.394188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394168 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 02 16:54:32.394188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394177 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 02 16:54:32.394188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394182 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394192 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394198 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394211 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394217 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394228 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 02 16:54:32.394324 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.394234 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 02 16:54:32.396422 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.396390 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 02 16:54:32.396422 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.396424 2565 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 02 16:54:32.399536 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.399517 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-t68jx" Jul 02 16:54:32.400886 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.400872 2565 server.go:1268] "Started kubelet" Jul 02 16:54:32.400983 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.400959 2565 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 02 16:54:32.401054 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.401006 2565 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 02 16:54:32.401163 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.401138 2565 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 02 16:54:32.401860 ip-10-0-130-120 systemd[1]: Started Kubernetes Kubelet. Jul 02 16:54:32.403941 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.403923 2565 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-120.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 02 16:54:32.404067 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.404045 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 02 16:54:32.404148 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.404107 2565 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-120.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 02 16:54:32.404439 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.404324 2565 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 02 16:54:32.405943 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.405930 2565 server.go:310] "Adding debug handlers to kubelet server" Jul 02 16:54:32.408543 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.407692 2565 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-120.ec2.internal.18be874e31f8337d default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-120.ec2.internal,UID:ip-10-0-130-120.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-120.ec2.internal,},FirstTimestamp:2026-07-02 16:54:32.400843645 +0000 UTC m=+0.361192712,LastTimestamp:2026-07-02 16:54:32.400843645 +0000 UTC m=+0.361192712,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-120.ec2.internal,}" Jul 02 16:54:32.409852 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.409834 2565 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 02 16:54:32.412745 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.412730 2565 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 02 16:54:32.412819 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.412751 2565 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 02 16:54:32.412819 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.412798 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.413714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413480 2565 factory.go:55] Registering systemd factory Jul 02 16:54:32.413714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413635 2565 factory.go:223] Registration of the systemd container factory successfully Jul 02 16:54:32.413714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413497 2565 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 02 16:54:32.413714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413563 2565 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 02 16:54:32.413714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413668 2565 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 02 16:54:32.413964 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413785 2565 reconstruct.go:97] "Volume reconstruction finished" Jul 02 16:54:32.413964 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.413806 2565 reconciler.go:29] "Reconciler: start to sync state" Jul 02 16:54:32.414065 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414024 2565 factory.go:153] Registering CRI-O factory Jul 02 16:54:32.414065 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414040 2565 factory.go:223] Registration of the crio container factory successfully Jul 02 16:54:32.414174 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414144 2565 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 02 16:54:32.414174 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414170 2565 factory.go:103] Registering Raw factory Jul 02 16:54:32.414277 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414185 2565 manager.go:1196] Started watching for new ooms in manager Jul 02 16:54:32.414616 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.414600 2565 manager.go:349] Starting recovery of all containers Jul 02 16:54:32.421622 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.421603 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:32.423874 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.423856 2565 manager.go:354] Recovery completed Jul 02 16:54:32.424104 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.424087 2565 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-130-120.ec2.internal\" not found" node="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.429609 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.429592 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.431693 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.431679 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.431748 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.431708 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.431748 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.431720 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.432235 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432222 2565 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 02 16:54:32.432286 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432234 2565 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 02 16:54:32.432286 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432239 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.432286 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432250 2565 state_mem.go:36] "Initialized new in-memory state store" Jul 02 16:54:32.432875 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432862 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.432921 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432886 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.432921 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.432896 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.434429 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.434418 2565 policy_none.go:49] "None policy: Start" Jul 02 16:54:32.434464 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.434433 2565 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 02 16:54:32.434464 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.434443 2565 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 02 16:54:32.436376 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.436365 2565 policy_none.go:47] "Start" Jul 02 16:54:32.476879 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.476855 2565 manager.go:337] "Starting Device Plugin manager" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.476994 2565 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477009 2565 server.go:87] "Starting device plugin registration server" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477310 2565 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477323 2565 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477473 2565 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477583 2565 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.477591 2565 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.478077 2565 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.478138 2565 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.486653 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.487887 2565 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.487923 2565 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.487961 2565 kubelet.go:2448] "Starting kubelet main sync loop" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.487992 2565 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 02 16:54:32.498959 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.490156 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:32.578408 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.578340 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.579336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.579317 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.579443 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.579355 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.579443 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.579378 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.579443 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.579410 2565 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.587749 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.587729 2565 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.587845 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.587752 2565 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-120.ec2.internal\": node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.588943 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.588925 2565 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal"] Jul 02 16:54:32.588998 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.588989 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.590858 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.590840 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.590933 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.590873 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.590933 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.590889 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.593262 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.593250 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.593382 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.593366 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.593437 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.593402 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.594197 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594179 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.594267 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594207 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.594267 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594217 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.594968 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594945 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.595069 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594976 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.595069 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.594990 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.596483 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.596469 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.596536 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.596498 2565 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 02 16:54:32.597244 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.597226 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientMemory" Jul 02 16:54:32.597322 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.597252 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasNoDiskPressure" Jul 02 16:54:32.597322 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.597263 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeHasSufficientPID" Jul 02 16:54:32.603078 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.603061 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.615072 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.615050 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.615168 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.615079 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.615168 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.615106 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/404f9a43cfb35449789f47a2943d4dea-config\") pod \"kube-apiserver-proxy-ip-10-0-130-120.ec2.internal\" (UID: \"404f9a43cfb35449789f47a2943d4dea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.619725 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.619708 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-120.ec2.internal\" not found" node="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.624133 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.624099 2565 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-120.ec2.internal\" not found" node="ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.703944 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.703910 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.716000 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.715977 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.716179 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.716004 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.716179 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.716026 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/404f9a43cfb35449789f47a2943d4dea-config\") pod \"kube-apiserver-proxy-ip-10-0-130-120.ec2.internal\" (UID: \"404f9a43cfb35449789f47a2943d4dea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.716179 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.716082 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.716179 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.716100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f01d13ef24f80d0e32b801b4ab5033d7-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal\" (UID: \"f01d13ef24f80d0e32b801b4ab5033d7\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.716179 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.716082 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/404f9a43cfb35449789f47a2943d4dea-config\") pod \"kube-apiserver-proxy-ip-10-0-130-120.ec2.internal\" (UID: \"404f9a43cfb35449789f47a2943d4dea\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.804370 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.804332 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.905214 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:32.905149 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:32.922688 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.922661 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:32.928113 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:32.927898 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:33.006111 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.006072 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.106654 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.106626 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.207216 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.207110 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.307701 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.307662 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.324817 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.324786 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:33.336847 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.336825 2565 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 02 16:54:33.336972 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.336946 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 16:54:33.337008 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.336986 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 16:54:33.337008 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.336984 2565 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 02 16:54:33.404819 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.404763 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-01 16:49:33 +0000 UTC" deadline="2027-12-12 18:07:50.733353979 +0000 UTC" Jul 02 16:54:33.404819 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.404806 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12673h13m17.328551528s" Jul 02 16:54:33.408276 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.408252 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.509284 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:33.509224 2565 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-130-120.ec2.internal\" not found" Jul 02 16:54:33.517880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.517856 2565 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:33.554844 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:33.554796 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf01d13ef24f80d0e32b801b4ab5033d7.slice/crio-8cc14342fbd418a8b0a28cbae3906599cf930549578758a1ef31f20f0ff20861 WatchSource:0}: Error finding container 8cc14342fbd418a8b0a28cbae3906599cf930549578758a1ef31f20f0ff20861: Status 404 returned error can't find the container with id 8cc14342fbd418a8b0a28cbae3906599cf930549578758a1ef31f20f0ff20861 Jul 02 16:54:33.555381 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:33.555358 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod404f9a43cfb35449789f47a2943d4dea.slice/crio-c347471fdfa568f62db558403bf412e62ea7c9ef519f60eb571c966350f522e2 WatchSource:0}: Error finding container c347471fdfa568f62db558403bf412e62ea7c9ef519f60eb571c966350f522e2: Status 404 returned error can't find the container with id c347471fdfa568f62db558403bf412e62ea7c9ef519f60eb571c966350f522e2 Jul 02 16:54:33.560829 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.560810 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 16:54:33.613312 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.613285 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" Jul 02 16:54:33.628299 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.628279 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 16:54:33.629257 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.629244 2565 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" Jul 02 16:54:33.637786 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:33.637768 2565 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 02 16:54:34.141627 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.141593 2565 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:34.390135 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.389828 2565 apiserver.go:52] "Watching apiserver" Jul 02 16:54:34.397582 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.397558 2565 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 02 16:54:34.398431 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.398000 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal","openshift-multus/network-metrics-daemon-g7wz9","openshift-network-diagnostics/network-check-target-x2vnt","openshift-network-operator/iptables-alerter-7fcvt","openshift-ovn-kubernetes/ovnkube-node-bzchs","kube-system/konnectivity-agent-smpjh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc","openshift-dns/node-resolver-rqqbn","openshift-image-registry/node-ca-d2hr6","openshift-multus/multus-additional-cni-plugins-w29hl","openshift-multus/multus-ld74z","kube-system/global-pull-secret-syncer-qv2h5","kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal","openshift-cluster-node-tuning-operator/tuned-sjkg9"] Jul 02 16:54:34.400811 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.400790 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.403631 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.403186 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 02 16:54:34.403631 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.403216 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-d6c48\"" Jul 02 16:54:34.403631 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.403489 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.403631 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.403514 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.403920 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.403760 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.405911 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.405880 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.406295 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.406271 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 02 16:54:34.406490 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.406476 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.406719 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.406703 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 02 16:54:34.407011 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.406988 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-jh742\"" Jul 02 16:54:34.407105 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.407027 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:34.407105 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.407090 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:34.409953 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.409927 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.412465 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.412444 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 02 16:54:34.412726 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.412710 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.412876 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.412856 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.413055 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.413028 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9bjq9\"" Jul 02 16:54:34.413281 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.413036 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.414154 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.414134 2565 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 02 16:54:34.415099 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.415076 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.416423 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.416406 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.416779 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.416748 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 02 16:54:34.416915 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.416899 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.417105 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.417080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 02 16:54:34.417985 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.417962 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 02 16:54:34.418670 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.418130 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.420012 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.419987 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-kqzbr\"" Jul 02 16:54:34.422683 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.422481 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.424485 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.424080 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-p8lwb\"" Jul 02 16:54:34.424485 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.424343 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 02 16:54:34.424485 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.424379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.424485 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.424462 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:34.424652 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.424611 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 02 16:54:34.425304 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.425102 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.425304 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.425226 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 02 16:54:34.425823 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.425581 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-hr462\"" Jul 02 16:54:34.425823 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.425707 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.426111 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.426458 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-mqr9x\"" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.426663 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.426804 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427375 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fb4wd\" (UniqueName: \"kubernetes.io/projected/02724429-317a-4f29-a028-2235b50f3cdb-kube-api-access-fb4wd\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.427459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427415 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427492 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d478d47-4b25-4ce7-8296-d8c75ff197e2-host-slash\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427531 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jsln9\" (UniqueName: \"kubernetes.io/projected/6d478d47-4b25-4ce7-8296-d8c75ff197e2-kube-api-access-jsln9\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427563 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-systemd-units\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427598 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-netns\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427632 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-systemd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-ovn\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427687 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-env-overrides\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91d5fcb7-2ac0-4139-8595-dcd0eef80101-agent-certs\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427793 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427858 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-kubelet\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427916 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-multus-daemon-config\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.427957 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428006 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428040 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-device-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428067 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428115 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-cni-binary-copy\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428161 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-k8s-cni-cncf-io\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428212 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-multus\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428243 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-script-lib\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428288 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-socket-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-os-release\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428373 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-bin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428398 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-node-log\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428431 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-netd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428496 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-sys-fs\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428550 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-netns\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428593 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-conf-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428625 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428656 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-registration-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.428704 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428688 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-system-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428740 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-socket-dir-parent\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428773 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-multus-certs\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-kubelet\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428847 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-var-lib-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.428903 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-etc-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.429664 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.429005 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-config\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431637 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.429053 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l7rlc\" (UniqueName: \"kubernetes.io/projected/e143c206-9ce6-40ec-bf52-9e0b0070278b-kube-api-access-l7rlc\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.431738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d478d47-4b25-4ce7-8296-d8c75ff197e2-iptables-alerter-script\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.431738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431692 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-slash\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431717 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-log-socket\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431734 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-bin\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431749 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovn-node-metrics-cert\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431767 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2gxsf\" (UniqueName: \"kubernetes.io/projected/046db9f0-94ed-46b7-ba2c-b80bc25212a4-kube-api-access-2gxsf\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91d5fcb7-2ac0-4139-8595-dcd0eef80101-konnectivity-ca\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431814 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-cnibin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431828 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-hostroot\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.431961 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.431842 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-etc-kubernetes\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.432142 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.432077 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.432545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.432289 2565 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 02 16:54:34.436361 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.436339 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 02 16:54:34.436755 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.436727 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-pqpd2\"" Jul 02 16:54:34.436898 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.436850 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 02 16:54:34.436898 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.436894 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.441310 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.438616 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.441310 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.438662 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:34.441310 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.439120 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-k6jgw\"" Jul 02 16:54:34.441310 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.439263 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 02 16:54:34.441310 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.439498 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 02 16:54:34.450259 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.450238 2565 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-q2ccj" Jul 02 16:54:34.460316 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.460293 2565 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-q2ccj" Jul 02 16:54:34.495459 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.495343 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" event={"ID":"404f9a43cfb35449789f47a2943d4dea","Type":"ContainerStarted","Data":"c347471fdfa568f62db558403bf412e62ea7c9ef519f60eb571c966350f522e2"} Jul 02 16:54:34.497182 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.497032 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" event={"ID":"f01d13ef24f80d0e32b801b4ab5033d7","Type":"ContainerStarted","Data":"8cc14342fbd418a8b0a28cbae3906599cf930549578758a1ef31f20f0ff20861"} Jul 02 16:54:34.515611 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.515587 2565 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 02 16:54:34.532025 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.531990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-socket-dir-parent\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532044 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-multus-certs\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-kubelet\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532079 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-socket-dir-parent\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532095 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-etc-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-config\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532141 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e80c9d4-a76a-46f4-9554-c258290c37b4-serviceca\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-dbus\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532171 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-etc-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532171 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-kubelet\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532196 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-multus-certs\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.532222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-slash\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532248 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-log-socket\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532255 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-slash\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-bin\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532312 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-log-socket\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532320 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-bin\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-systemd\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532346 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-host\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532371 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-binary-copy\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532416 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532462 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d478d47-4b25-4ce7-8296-d8c75ff197e2-host-slash\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532539 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91d5fcb7-2ac0-4139-8595-dcd0eef80101-agent-certs\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532582 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6d478d47-4b25-4ce7-8296-d8c75ff197e2-host-slash\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532631 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532804 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-env-overrides\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532882 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysconfig\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.532932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532907 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532953 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-hosts-file\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.532970 2565 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533068 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5g4d7\" (UniqueName: \"kubernetes.io/projected/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-kube-api-access-5g4d7\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-multus-daemon-config\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533100 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-config\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533114 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533163 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-modprobe-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533180 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-conf\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533250 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-kubelet-config\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533295 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533298 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533338 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533393 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-cni-binary-copy\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-k8s-cni-cncf-io\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533477 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m7qrl\" (UniqueName: \"kubernetes.io/projected/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-kube-api-access-m7qrl\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533504 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-socket-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.533575 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533540 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-os-release\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533567 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-bin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533592 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80c9d4-a76a-46f4-9554-c258290c37b4-host\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wpq2\" (UniqueName: \"kubernetes.io/projected/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-kube-api-access-2wpq2\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533681 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x7gww\" (UniqueName: \"kubernetes.io/projected/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-kube-api-access-x7gww\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533712 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-tmp-dir\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-sys-fs\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533739 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-env-overrides\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533770 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-netns\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-conf-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533812 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533829 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-run\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533854 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-lib-modules\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533926 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-registration-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-cni-binary-copy\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533946 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-system-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533985 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-k8s-cni-cncf-io\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.534564 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.533987 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-var-lib-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534021 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534028 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovn-node-metrics-cert\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534055 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2gxsf\" (UniqueName: \"kubernetes.io/projected/046db9f0-94ed-46b7-ba2c-b80bc25212a4-kube-api-access-2gxsf\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534074 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-sys\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534118 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l7rlc\" (UniqueName: \"kubernetes.io/projected/e143c206-9ce6-40ec-bf52-9e0b0070278b-kube-api-access-l7rlc\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534159 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d478d47-4b25-4ce7-8296-d8c75ff197e2-iptables-alerter-script\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-os-release\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534242 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91d5fcb7-2ac0-4139-8595-dcd0eef80101-konnectivity-ca\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qrpp4\" (UniqueName: \"kubernetes.io/projected/4e80c9d4-a76a-46f4-9554-c258290c37b4-kube-api-access-qrpp4\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534328 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-socket-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534330 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-kubernetes\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-os-release\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534393 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-bin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534436 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-cnibin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534461 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-hostroot\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.535432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534510 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-etc-kubernetes\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534522 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/02724429-317a-4f29-a028-2235b50f3cdb-multus-daemon-config\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534532 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fb4wd\" (UniqueName: \"kubernetes.io/projected/02724429-317a-4f29-a028-2235b50f3cdb-kube-api-access-fb4wd\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534618 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-system-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.534978 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535001 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-hostroot\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535063 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-sys-fs\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535137 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-run-netns\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535171 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-conf-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535261 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-registration-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535311 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-cnibin\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-etc-kubernetes\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jsln9\" (UniqueName: \"kubernetes.io/projected/6d478d47-4b25-4ce7-8296-d8c75ff197e2-kube-api-access-jsln9\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535549 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-ovn\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535577 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-systemd-units\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535603 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-netns\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-systemd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535701 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-tuned\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.536261 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535770 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535807 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535850 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-kubelet\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535883 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/91d5fcb7-2ac0-4139-8595-dcd0eef80101-konnectivity-ca\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535890 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-system-cni-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.535922 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-device-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536013 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-kubelet\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536050 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-multus\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536070 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-script-lib\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536098 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-systemd-units\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536141 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-systemd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536153 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-run-ovn\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-var-lib-kubelet\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536271 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-netns\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-node-log\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536353 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536459 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-var-lib-openvswitch\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536623 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-netd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-tmp\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536696 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cnibin\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536685 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-etc-selinux\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536806 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-host-var-lib-cni-multus\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536810 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/6d478d47-4b25-4ce7-8296-d8c75ff197e2-iptables-alerter-script\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536841 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovnkube-script-lib\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536872 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/e143c206-9ce6-40ec-bf52-9e0b0070278b-device-dir\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536824 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/02724429-317a-4f29-a028-2235b50f3cdb-multus-cni-dir\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536889 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-run-ovn-kubernetes\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.536952 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-node-log\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.537007 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/046db9f0-94ed-46b7-ba2c-b80bc25212a4-host-cni-netd\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.537891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.537750 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/91d5fcb7-2ac0-4139-8595-dcd0eef80101-agent-certs\") pod \"konnectivity-agent-smpjh\" (UID: \"91d5fcb7-2ac0-4139-8595-dcd0eef80101\") " pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.538377 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.538086 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/046db9f0-94ed-46b7-ba2c-b80bc25212a4-ovn-node-metrics-cert\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.538585 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.538560 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:34.538639 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.538587 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:34.538639 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.538602 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:34.538736 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.538667 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:35.038647439 +0000 UTC m=+2.998996498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:34.544969 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.544941 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fb4wd\" (UniqueName: \"kubernetes.io/projected/02724429-317a-4f29-a028-2235b50f3cdb-kube-api-access-fb4wd\") pod \"multus-ld74z\" (UID: \"02724429-317a-4f29-a028-2235b50f3cdb\") " pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.545621 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.545577 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l7rlc\" (UniqueName: \"kubernetes.io/projected/e143c206-9ce6-40ec-bf52-9e0b0070278b-kube-api-access-l7rlc\") pod \"aws-ebs-csi-driver-node-qzbjc\" (UID: \"e143c206-9ce6-40ec-bf52-9e0b0070278b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.546520 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.546496 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2gxsf\" (UniqueName: \"kubernetes.io/projected/046db9f0-94ed-46b7-ba2c-b80bc25212a4-kube-api-access-2gxsf\") pod \"ovnkube-node-bzchs\" (UID: \"046db9f0-94ed-46b7-ba2c-b80bc25212a4\") " pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.548019 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.547991 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jsln9\" (UniqueName: \"kubernetes.io/projected/6d478d47-4b25-4ce7-8296-d8c75ff197e2-kube-api-access-jsln9\") pod \"iptables-alerter-7fcvt\" (UID: \"6d478d47-4b25-4ce7-8296-d8c75ff197e2\") " pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637022 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysconfig\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637068 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637097 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-hosts-file\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637202 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-hosts-file\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.637230 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637249 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5g4d7\" (UniqueName: \"kubernetes.io/projected/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-kube-api-access-5g4d7\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-modprobe-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-conf\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637314 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-kubelet-config\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.637342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637335 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m7qrl\" (UniqueName: \"kubernetes.io/projected/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-kube-api-access-m7qrl\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80c9d4-a76a-46f4-9554-c258290c37b4-host\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637410 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2wpq2\" (UniqueName: \"kubernetes.io/projected/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-kube-api-access-2wpq2\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637452 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysconfig\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637451 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x7gww\" (UniqueName: \"kubernetes.io/projected/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-kube-api-access-x7gww\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637489 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-tmp-dir\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637509 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-run\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637524 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-lib-modules\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637542 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-sys\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637562 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qrpp4\" (UniqueName: \"kubernetes.io/projected/4e80c9d4-a76a-46f4-9554-c258290c37b4-kube-api-access-qrpp4\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637605 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-kubernetes\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637633 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-os-release\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637650 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637690 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-tuned\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637707 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.637739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-system-cni-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637753 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-var-lib-kubelet\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637764 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-kubelet-config\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637777 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-tmp\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637794 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cnibin\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e80c9d4-a76a-46f4-9554-c258290c37b4-serviceca\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.637847 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637865 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-dbus\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.637893 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:54:35.137876508 +0000 UTC m=+3.098225564 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637895 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-modprobe-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-systemd\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:34.637939 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:35.137919245 +0000 UTC m=+3.098268309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637960 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-host\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637993 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-binary-copy\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638019 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/4e80c9d4-a76a-46f4-9554-c258290c37b4-host\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638039 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.638538 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638067 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638253 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-tmp-dir\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638452 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-kubernetes\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638506 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-system-cni-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638453 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-os-release\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638541 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cnibin\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638561 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-lib-modules\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638572 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-var-lib-kubelet\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638593 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-sys\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638631 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-run\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.637963 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-systemd\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638690 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/b801cb53-4e0c-4a93-8a19-49f2a550187f-dbus\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638759 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-conf\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638782 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638783 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638796 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-host\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638913 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-sysctl-d\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.638965 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/4e80c9d4-a76a-46f4-9554-c258290c37b4-serviceca\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.639305 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.639313 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-binary-copy\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.640106 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.639402 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.641391 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.641330 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-etc-tuned\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.641827 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.641807 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-tmp\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.649739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.649710 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x7gww\" (UniqueName: \"kubernetes.io/projected/14f5451f-a4c8-4cc2-9457-4e12e2ccb347-kube-api-access-x7gww\") pod \"multus-additional-cni-plugins-w29hl\" (UID: \"14f5451f-a4c8-4cc2-9457-4e12e2ccb347\") " pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.651192 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.651168 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wpq2\" (UniqueName: \"kubernetes.io/projected/ad6c85d5-6fc0-455f-aa3f-32a9b907972b-kube-api-access-2wpq2\") pod \"tuned-sjkg9\" (UID: \"ad6c85d5-6fc0-455f-aa3f-32a9b907972b\") " pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.651583 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.651565 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m7qrl\" (UniqueName: \"kubernetes.io/projected/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-kube-api-access-m7qrl\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:34.651731 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.651710 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qrpp4\" (UniqueName: \"kubernetes.io/projected/4e80c9d4-a76a-46f4-9554-c258290c37b4-kube-api-access-qrpp4\") pod \"node-ca-d2hr6\" (UID: \"4e80c9d4-a76a-46f4-9554-c258290c37b4\") " pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.651794 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.651711 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5g4d7\" (UniqueName: \"kubernetes.io/projected/25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d-kube-api-access-5g4d7\") pod \"node-resolver-rqqbn\" (UID: \"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d\") " pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.716747 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.716706 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" Jul 02 16:54:34.733355 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.733332 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-ld74z" Jul 02 16:54:34.743477 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.743450 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-7fcvt" Jul 02 16:54:34.752428 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.752401 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:54:34.761499 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.761466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:34.768231 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.768211 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-d2hr6" Jul 02 16:54:34.778115 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.778098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" Jul 02 16:54:34.787986 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.787967 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w29hl" Jul 02 16:54:34.796935 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.796913 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-rqqbn" Jul 02 16:54:34.923823 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:34.923729 2565 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:35.041870 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:35.041823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:35.042049 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.042022 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:35.042049 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.042046 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:35.042161 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.042061 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:35.042161 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.042150 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.042113986 +0000 UTC m=+4.002463045 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:35.143119 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:35.143073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:35.143343 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:35.143140 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:35.143343 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.143265 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:35.143343 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.143289 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:35.143343 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.143341 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.143322437 +0000 UTC m=+4.103671492 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:35.143586 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:35.143366 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:54:36.143352527 +0000 UTC m=+4.103701584 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:35.466864 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:35.466827 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 16:49:35 +0000 UTC" deadline="2028-03-13 20:57:18.73437771 +0000 UTC" Jul 02 16:54:35.466864 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:35.466858 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14884h2m43.267522791s" Jul 02 16:54:35.601220 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:35.601185 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91d5fcb7_2ac0_4139_8595_dcd0eef80101.slice/crio-10d6dccdd0366f307d25c8ae6086d7e752f3d2c166cea3b0e309994c0c3428fa WatchSource:0}: Error finding container 10d6dccdd0366f307d25c8ae6086d7e752f3d2c166cea3b0e309994c0c3428fa: Status 404 returned error can't find the container with id 10d6dccdd0366f307d25c8ae6086d7e752f3d2c166cea3b0e309994c0c3428fa Jul 02 16:54:35.602768 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:35.602738 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod046db9f0_94ed_46b7_ba2c_b80bc25212a4.slice/crio-d3f3f1d666b0e03cb558605060ef0f44ceecd8e06e90e70825cab46793a4d00d WatchSource:0}: Error finding container d3f3f1d666b0e03cb558605060ef0f44ceecd8e06e90e70825cab46793a4d00d: Status 404 returned error can't find the container with id d3f3f1d666b0e03cb558605060ef0f44ceecd8e06e90e70825cab46793a4d00d Jul 02 16:54:35.603581 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:35.603560 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e80c9d4_a76a_46f4_9554_c258290c37b4.slice/crio-60d6f3bf592a5211f59cb255f35ac650a4368fc67f9df2a2eb3b6ad3ae2007bc WatchSource:0}: Error finding container 60d6f3bf592a5211f59cb255f35ac650a4368fc67f9df2a2eb3b6ad3ae2007bc: Status 404 returned error can't find the container with id 60d6f3bf592a5211f59cb255f35ac650a4368fc67f9df2a2eb3b6ad3ae2007bc Jul 02 16:54:35.628749 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:35.628667 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02724429_317a_4f29_a028_2235b50f3cdb.slice/crio-c19e54e86870c4944ffaefd2badd6dfbf95c16a6d0bd230ca43307853b7f55a1 WatchSource:0}: Error finding container c19e54e86870c4944ffaefd2badd6dfbf95c16a6d0bd230ca43307853b7f55a1: Status 404 returned error can't find the container with id c19e54e86870c4944ffaefd2badd6dfbf95c16a6d0bd230ca43307853b7f55a1 Jul 02 16:54:35.646967 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:54:35.646936 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6d478d47_4b25_4ce7_8296_d8c75ff197e2.slice/crio-6a2e2c031ddfb4d2a8036ddcd93492a05f9f5851ce9145ecae9567cca35d3e11 WatchSource:0}: Error finding container 6a2e2c031ddfb4d2a8036ddcd93492a05f9f5851ce9145ecae9567cca35d3e11: Status 404 returned error can't find the container with id 6a2e2c031ddfb4d2a8036ddcd93492a05f9f5851ce9145ecae9567cca35d3e11 Jul 02 16:54:36.050462 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.050382 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:36.050603 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.050565 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:36.050603 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.050584 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:36.050603 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.050593 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:36.050721 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.050644 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:38.05063014 +0000 UTC m=+6.010979196 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.151014 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.151071 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.151272 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.151343 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:54:38.151324122 +0000 UTC m=+6.111673179 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.151759 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:36.151865 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.151821 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:38.151803227 +0000 UTC m=+6.112152297 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:36.467116 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.466989 2565 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-01 16:49:35 +0000 UTC" deadline="2028-04-09 06:12:08.922995606 +0000 UTC" Jul 02 16:54:36.467116 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.467057 2565 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15517h17m32.45594329s" Jul 02 16:54:36.490690 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.490632 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:36.490865 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.490702 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:36.490865 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.490713 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:36.490865 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.490800 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:36.491255 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.491049 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:36.491255 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:36.491205 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:36.497594 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.497567 2565 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 02 16:54:36.508384 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.508351 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" event={"ID":"404f9a43cfb35449789f47a2943d4dea","Type":"ContainerStarted","Data":"654019df10fea83dc88f1725bc8dbaa24002c06c1cc31234131ffc346c653faa"} Jul 02 16:54:36.519700 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.519666 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7fcvt" event={"ID":"6d478d47-4b25-4ce7-8296-d8c75ff197e2","Type":"ContainerStarted","Data":"6a2e2c031ddfb4d2a8036ddcd93492a05f9f5851ce9145ecae9567cca35d3e11"} Jul 02 16:54:36.528365 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.528337 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqqbn" event={"ID":"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d","Type":"ContainerStarted","Data":"fecbd9fb90c2838f87f095ab71ac2b984bb55a3c227a969b01721625b8b20edf"} Jul 02 16:54:36.542163 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.542126 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" event={"ID":"e143c206-9ce6-40ec-bf52-9e0b0070278b","Type":"ContainerStarted","Data":"0851a68af5dafcf974fa6676f81f1a98795a615713205465b4be6699298c70fa"} Jul 02 16:54:36.546247 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.546211 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" event={"ID":"ad6c85d5-6fc0-455f-aa3f-32a9b907972b","Type":"ContainerStarted","Data":"9ef6eb994f31d2774bd43b71ad95959808d29c28bd4cd5d69568747ab9d596bc"} Jul 02 16:54:36.549150 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.549060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ld74z" event={"ID":"02724429-317a-4f29-a028-2235b50f3cdb","Type":"ContainerStarted","Data":"c19e54e86870c4944ffaefd2badd6dfbf95c16a6d0bd230ca43307853b7f55a1"} Jul 02 16:54:36.551852 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.551755 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"d3f3f1d666b0e03cb558605060ef0f44ceecd8e06e90e70825cab46793a4d00d"} Jul 02 16:54:36.555451 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.555320 2565 generic.go:356] "Generic (PLEG): container finished" podID="f01d13ef24f80d0e32b801b4ab5033d7" containerID="ecaa9048c547ad5f236ce948f6ef737e7839368d0995f8e1c191ee53024744b9" exitCode=0 Jul 02 16:54:36.555451 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.555405 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" event={"ID":"f01d13ef24f80d0e32b801b4ab5033d7","Type":"ContainerDied","Data":"ecaa9048c547ad5f236ce948f6ef737e7839368d0995f8e1c191ee53024744b9"} Jul 02 16:54:36.560806 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.560738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerStarted","Data":"c0a329a06072387174bdbfb8aaa8fc100f29d93d5ce457fc9a92b46baaf8ba89"} Jul 02 16:54:36.570519 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.570459 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-120.ec2.internal" podStartSLOduration=2.570437078 podStartE2EDuration="2.570437078s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:36.522835517 +0000 UTC m=+4.483184598" watchObservedRunningTime="2026-07-02 16:54:36.570437078 +0000 UTC m=+4.530786272" Jul 02 16:54:36.576503 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.576427 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d2hr6" event={"ID":"4e80c9d4-a76a-46f4-9554-c258290c37b4","Type":"ContainerStarted","Data":"60d6f3bf592a5211f59cb255f35ac650a4368fc67f9df2a2eb3b6ad3ae2007bc"} Jul 02 16:54:36.582934 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:36.582900 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-smpjh" event={"ID":"91d5fcb7-2ac0-4139-8595-dcd0eef80101","Type":"ContainerStarted","Data":"10d6dccdd0366f307d25c8ae6086d7e752f3d2c166cea3b0e309994c0c3428fa"} Jul 02 16:54:37.610368 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:37.610259 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" event={"ID":"f01d13ef24f80d0e32b801b4ab5033d7","Type":"ContainerStarted","Data":"845048625fa57809053883044e35e219db347322953c20e9a0751b90fa90f949"} Jul 02 16:54:37.626321 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:37.626262 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-120.ec2.internal" podStartSLOduration=3.626230926 podStartE2EDuration="3.626230926s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:37.626199679 +0000 UTC m=+5.586548759" watchObservedRunningTime="2026-07-02 16:54:37.626230926 +0000 UTC m=+5.586580007" Jul 02 16:54:38.070335 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.070246 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:38.070512 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.070447 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:38.070512 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.070470 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:38.070512 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.070484 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:38.070700 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.070550 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:42.070529102 +0000 UTC m=+10.030878179 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:38.170773 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.170730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:38.170972 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.170791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:38.170972 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.170966 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:38.171102 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.171033 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:54:42.171013698 +0000 UTC m=+10.131362755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:38.171501 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.171480 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:38.171597 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.171541 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:42.171522382 +0000 UTC m=+10.131871446 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:38.489396 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.489303 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:38.489575 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.489476 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:38.489924 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.489905 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:38.490023 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.490002 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:38.492626 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:38.492372 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:38.492626 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:38.492472 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:40.491377 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:40.490644 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:40.491377 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:40.490780 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:40.491377 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:40.491180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:40.491377 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:40.491321 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:40.492298 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:40.492104 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:40.492298 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:40.492222 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:42.105133 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.105059 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:42.105582 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.105328 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:42.105582 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.105353 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:42.105582 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.105368 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:42.105582 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.105430 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:50.105411042 +0000 UTC m=+18.065760104 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:42.206008 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.205962 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:42.206231 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.206024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:42.206231 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.206101 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:42.206231 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.206200 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:54:50.206179749 +0000 UTC m=+18.166528813 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:42.206574 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.206106 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:42.206700 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.206612 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:54:50.206593531 +0000 UTC m=+18.166942589 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:42.493832 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.493756 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:42.493994 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.493897 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:42.495583 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.494359 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:42.495583 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.494462 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:42.495583 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:42.494516 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:42.495583 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:42.494579 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:44.492421 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:44.492386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:44.492931 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:44.492526 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:44.492931 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:44.492918 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:44.493064 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:44.493036 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:44.493144 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:44.493086 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:44.493144 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:44.493134 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:46.491875 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:46.491842 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:46.492345 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:46.491881 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:46.492345 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:46.491846 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:46.492345 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:46.491966 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:46.492345 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:46.492045 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:46.492345 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:46.492138 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:48.488972 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:48.488939 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:48.489467 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:48.488941 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:48.489467 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:48.489073 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:48.489467 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:48.489104 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:48.489467 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:48.488950 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:48.489467 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:48.489195 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:50.167733 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.167689 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:50.168240 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.167883 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:54:50.168240 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.167907 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:54:50.168240 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.167922 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:50.168240 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.167991 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:06.167970693 +0000 UTC m=+34.128319752 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:54:50.268744 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.268706 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:50.268916 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.268756 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:50.268916 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.268904 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:50.269026 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.268911 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:50.269026 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.268984 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:06.268968243 +0000 UTC m=+34.229317300 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:54:50.269026 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.269003 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:55:06.268993854 +0000 UTC m=+34.229342911 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:54:50.488984 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.488912 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:50.488984 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.488952 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:50.488984 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:50.488913 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:50.489266 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.489034 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:50.489266 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.489151 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:50.489266 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:50.489230 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:52.491140 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:52.490910 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:52.491593 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:52.490966 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:52.491593 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:52.491240 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:52.491593 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:52.490969 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:52.491593 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:52.491319 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:52.491593 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:52.491416 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:54.488744 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:54.488707 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:54.488744 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:54.488729 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:54.489269 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:54.488717 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:54.489269 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:54.488842 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:54.489269 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:54.488904 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:54.489269 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:54.488955 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:55.648604 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.648573 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" event={"ID":"ad6c85d5-6fc0-455f-aa3f-32a9b907972b","Type":"ContainerStarted","Data":"50376691b6d111da7864915bdacf6ca3bfb7c9a6322386655b6ebf062214a394"} Jul 02 16:54:55.651926 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.651897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-ld74z" event={"ID":"02724429-317a-4f29-a028-2235b50f3cdb","Type":"ContainerStarted","Data":"187ebf9859c26512e4a68b3f7800f8aff41d11d45d283705673a6c2a042de017"} Jul 02 16:54:55.654155 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.654109 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"1ef030a61db14c50c6f960e1ca0dbdd2241a2a1ff188be916c6324163484ce6c"} Jul 02 16:54:55.655485 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.655448 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerStarted","Data":"a6bceb4869983c0b2e09430f6c1a06a35f4a155b53af3b3448034da50eaeb6c6"} Jul 02 16:54:55.656884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.656864 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-d2hr6" event={"ID":"4e80c9d4-a76a-46f4-9554-c258290c37b4","Type":"ContainerStarted","Data":"e83f118d24794b959d49a42800abd36e48f8b310fd956a9576f99e8614392610"} Jul 02 16:54:55.662687 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.662663 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-smpjh" event={"ID":"91d5fcb7-2ac0-4139-8595-dcd0eef80101","Type":"ContainerStarted","Data":"b3327cd01d63702f279381fdf80ee2666c8aa62826bceae4acbcca5c6aff765a"} Jul 02 16:54:55.664181 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.664161 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-rqqbn" event={"ID":"25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d","Type":"ContainerStarted","Data":"9251c2c717d4ab3e64a20e0cdb7d36d5bbf5b3b8207c6e30ec925e8ea2fb0779"} Jul 02 16:54:55.665623 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.665602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" event={"ID":"e143c206-9ce6-40ec-bf52-9e0b0070278b","Type":"ContainerStarted","Data":"13fb54563dca39b133a5fc4344334bb45d61b6ba0848e8f33c5d63b397728f31"} Jul 02 16:54:55.678433 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.678288 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sjkg9" podStartSLOduration=2.906677652 podStartE2EDuration="22.678273054s" totalImagesPullingTime="19.771595402s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.664851603 +0000 UTC m=+23.625200679" watchObservedRunningTime="2026-07-02 16:54:55.678273054 +0000 UTC m=+23.638622178" Jul 02 16:54:55.679146 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.678863 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-d2hr6" podStartSLOduration=2.912605563 podStartE2EDuration="22.678855328s" totalImagesPullingTime="19.766249765s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.678608135 +0000 UTC m=+23.638957213" watchObservedRunningTime="2026-07-02 16:54:55.678855328 +0000 UTC m=+23.639204403" Jul 02 16:54:55.716977 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.716928 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-ld74z" podStartSLOduration=2.900779708 podStartE2EDuration="22.716905905s" totalImagesPullingTime="19.816126197s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.715784844 +0000 UTC m=+23.676133932" watchObservedRunningTime="2026-07-02 16:54:55.716905905 +0000 UTC m=+23.677254982" Jul 02 16:54:55.733675 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.733628 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-smpjh" podStartSLOduration=2.992085868 podStartE2EDuration="22.733614234s" totalImagesPullingTime="19.741528366s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.733158663 +0000 UTC m=+23.693507758" watchObservedRunningTime="2026-07-02 16:54:55.733614234 +0000 UTC m=+23.693963309" Jul 02 16:54:55.746854 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:55.746803 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-rqqbn" podStartSLOduration=2.999428128 podStartE2EDuration="22.746785527s" totalImagesPullingTime="19.747357399s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:55.746326036 +0000 UTC m=+23.706675139" watchObservedRunningTime="2026-07-02 16:54:55.746785527 +0000 UTC m=+23.707134603" Jul 02 16:54:56.488720 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.488493 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:56.488984 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.488501 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:56.488984 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:56.488823 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:56.488984 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.488556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:56.488984 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:56.488920 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:56.488984 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:56.488953 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:56.670023 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.669954 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670292 2565 generic.go:356] "Generic (PLEG): container finished" podID="046db9f0-94ed-46b7-ba2c-b80bc25212a4" containerID="66f58b86ed5d02e26923a4dd45c9ccb88705b59197c61d3674c3ca8cdc6ae944" exitCode=1 Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670348 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"a719e8f29c57e5a4686dfc1f3995107fc6519e6c0fee6b8a9f7f8e833c54d9b7"} Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670413 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"04be7ebf42b880cdd7f69a64d4878ca0c0ecb3d5cc6f3b73307dabe58b35ec7a"} Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670446 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"b734dec674704e84ad620ec240f72d56598e98b4f04b048fba1e4954ee50ae7c"} Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670478 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"2bac631f8499e3e3ac995fa580d23e60e1625ad53c782124fc07c9e8ddfbd10f"} Jul 02 16:54:56.670545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.670505 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerDied","Data":"66f58b86ed5d02e26923a4dd45c9ccb88705b59197c61d3674c3ca8cdc6ae944"} Jul 02 16:54:56.671804 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.671781 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="a6bceb4869983c0b2e09430f6c1a06a35f4a155b53af3b3448034da50eaeb6c6" exitCode=0 Jul 02 16:54:56.671902 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.671867 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"a6bceb4869983c0b2e09430f6c1a06a35f4a155b53af3b3448034da50eaeb6c6"} Jul 02 16:54:56.991302 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:56.991154 2565 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 02 16:54:57.492740 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.492004 2565 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-02T16:54:56.99128599Z","UUID":"8397b1b8-885a-4343-9928-61c06ace8d4c","Handler":null,"Name":"","Endpoint":""} Jul 02 16:54:57.493780 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.493755 2565 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 02 16:54:57.493916 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.493802 2565 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 02 16:54:57.676481 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.676444 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" event={"ID":"e143c206-9ce6-40ec-bf52-9e0b0070278b","Type":"ContainerStarted","Data":"6ce72473e6f60e25ebdd4d05be941c1bccd5ac5652f504fb6df5550ade3952e7"} Jul 02 16:54:57.678163 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.678111 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-7fcvt" event={"ID":"6d478d47-4b25-4ce7-8296-d8c75ff197e2","Type":"ContainerStarted","Data":"8a03d93677b6b6f08f27ca2fdfdf81f4ed51c0921d935b8ac4a2e5139ea5bce4"} Jul 02 16:54:57.693765 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:57.693715 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-7fcvt" podStartSLOduration=4.945423432 podStartE2EDuration="24.693695516s" totalImagesPullingTime="19.748272084s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:57.692862946 +0000 UTC m=+25.653212216" watchObservedRunningTime="2026-07-02 16:54:57.693695516 +0000 UTC m=+25.654044596" Jul 02 16:54:58.135069 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.135043 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:58.136361 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.136340 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:58.488545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.488466 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:54:58.488545 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.488499 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:54:58.488761 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:58.488582 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:54:58.488761 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.488467 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:54:58.488761 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:58.488663 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:54:58.488761 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:54:58.488732 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:54:58.682581 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.682549 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" event={"ID":"e143c206-9ce6-40ec-bf52-9e0b0070278b","Type":"ContainerStarted","Data":"82e7aa18c99c04df117615781f5e8ce2a40d50e365313e7420ac3b9d28b0f4a5"} Jul 02 16:54:58.687994 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.687953 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:54:58.688427 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.688401 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"32a4af2b453a1f4eec2895dea1b34dcbe0fb9e793f08c7874f36eaefcfe5d353"} Jul 02 16:54:58.688673 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.688655 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:58.689148 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.689113 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-smpjh" Jul 02 16:54:58.700444 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:54:58.700399 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-qzbjc" podStartSLOduration=3.600838931 podStartE2EDuration="25.700383535s" totalImagesPullingTime="22.099544604s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:54:58.700199174 +0000 UTC m=+26.660548249" watchObservedRunningTime="2026-07-02 16:54:58.700383535 +0000 UTC m=+26.660732613" Jul 02 16:55:00.488837 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:00.488725 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:00.488837 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:00.488740 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:00.489445 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:00.488896 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:55:00.489445 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:00.488960 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:00.489445 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:00.489087 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:55:00.489445 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:00.489208 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:55:01.697807 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.697650 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:55:01.698401 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.698158 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"1549fb424876487d66f062d1423a67368d2c07a162908a187585135fe4059b50"} Jul 02 16:55:01.698401 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.698380 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:01.698579 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.698562 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:01.698643 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.698595 2565 scope.go:117] "RemoveContainer" containerID="66f58b86ed5d02e26923a4dd45c9ccb88705b59197c61d3674c3ca8cdc6ae944" Jul 02 16:55:01.698643 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.698629 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:01.700041 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.699944 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="79cc623f25e910d05823411ff401ba2ef9b618700ff8702e4efc2436ca5c0a43" exitCode=0 Jul 02 16:55:01.700041 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.699972 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"79cc623f25e910d05823411ff401ba2ef9b618700ff8702e4efc2436ca5c0a43"} Jul 02 16:55:01.715187 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.715168 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:01.715309 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:01.715295 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.490499 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.490665 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.490623 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.490806 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.490962 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:55:02.492939 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.491071 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:55:02.643944 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.643910 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qv2h5"] Jul 02 16:55:02.647440 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.647410 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x2vnt"] Jul 02 16:55:02.648096 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.648078 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g7wz9"] Jul 02 16:55:02.705319 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.705251 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:55:02.705821 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.705655 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:02.705821 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.705694 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" event={"ID":"046db9f0-94ed-46b7-ba2c-b80bc25212a4","Type":"ContainerStarted","Data":"1e0bd573399edba5c634a38b0a6964f08ec79417ebd9ef6cfb6a6857bfd928c0"} Jul 02 16:55:02.705821 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.705745 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:55:02.705935 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.705859 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:02.705971 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.705954 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:55:02.706012 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.705960 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:02.706072 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:02.706055 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:55:02.732215 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:02.732162 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" podStartSLOduration=9.883858103 podStartE2EDuration="29.732143037s" totalImagesPullingTime="19.848284934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:02.731499823 +0000 UTC m=+30.691848898" watchObservedRunningTime="2026-07-02 16:55:02.732143037 +0000 UTC m=+30.692492114" Jul 02 16:55:03.710252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:03.710217 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="7eb6283ac5bbd17aed5deb22ea9a64f4a84df52c474ff5962f259e0418bac53f" exitCode=0 Jul 02 16:55:03.710785 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:03.710324 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"7eb6283ac5bbd17aed5deb22ea9a64f4a84df52c474ff5962f259e0418bac53f"} Jul 02 16:55:04.488865 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:04.488833 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:04.488981 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:04.488941 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:55:04.488981 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:04.488839 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:04.489064 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:04.488832 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:04.489112 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:04.489001 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:55:04.489189 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:04.489113 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:55:05.719915 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:05.719676 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="ba98a8786ec77b80b898f56d329a9266cc5ad77ed869e818ea0d5c0558364864" exitCode=0 Jul 02 16:55:05.720391 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:05.719765 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"ba98a8786ec77b80b898f56d329a9266cc5ad77ed869e818ea0d5c0558364864"} Jul 02 16:55:06.193710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.193622 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:06.193832 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.193737 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 02 16:55:06.193832 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.193756 2565 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 02 16:55:06.193832 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.193769 2565 projected.go:196] Error preparing data for projected volume kube-api-access-xb7xm for pod openshift-network-diagnostics/network-check-target-x2vnt: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:55:06.193832 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.193819 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm podName:330b7496-741f-49eb-8d70-93c0111a44b2 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:38.193805762 +0000 UTC m=+66.154154817 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-xb7xm" (UniqueName: "kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm") pod "network-check-target-x2vnt" (UID: "330b7496-741f-49eb-8d70-93c0111a44b2") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 02 16:55:06.294507 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.294464 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:06.294507 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.294512 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:06.294746 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.294618 2565 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 02 16:55:06.294746 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.294620 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:55:06.294746 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.294680 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret podName:b801cb53-4e0c-4a93-8a19-49f2a550187f nodeName:}" failed. No retries permitted until 2026-07-02 16:55:38.294660832 +0000 UTC m=+66.255009887 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret") pod "global-pull-secret-syncer-qv2h5" (UID: "b801cb53-4e0c-4a93-8a19-49f2a550187f") : object "kube-system"/"original-pull-secret" not registered Jul 02 16:55:06.294746 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.294699 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:38.294688653 +0000 UTC m=+66.255037714 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 02 16:55:06.489252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.489009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:06.489252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.489009 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:06.489252 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.489132 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-qv2h5" podUID="b801cb53-4e0c-4a93-8a19-49f2a550187f" Jul 02 16:55:06.489252 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.489214 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:55:06.489252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:06.489240 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:06.489563 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:06.489318 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-x2vnt" podUID="330b7496-741f-49eb-8d70-93c0111a44b2" Jul 02 16:55:08.354719 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.354652 2565 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-130-120.ec2.internal" event="NodeReady" Jul 02 16:55:08.355045 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.354754 2565 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 02 16:55:08.397828 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.397801 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gfmwc"] Jul 02 16:55:08.399746 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.399732 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.401992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.401967 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 02 16:55:08.402238 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.402217 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 02 16:55:08.402429 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.402415 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmn65\"" Jul 02 16:55:08.407432 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.407409 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-bzpjp"] Jul 02 16:55:08.409081 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.409063 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.410405 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.410382 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gfmwc"] Jul 02 16:55:08.411608 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.411583 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 02 16:55:08.411712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.411596 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 02 16:55:08.411712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.411645 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 02 16:55:08.411712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.411676 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-947jv\"" Jul 02 16:55:08.421401 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.421381 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzpjp"] Jul 02 16:55:08.491904 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.491885 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:08.492064 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.492049 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:08.492197 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.492180 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:08.494598 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 16:55:08.494707 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494596 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 16:55:08.494707 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494617 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 16:55:08.494707 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494631 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 16:55:08.494707 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494579 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-442wp\"" Jul 02 16:55:08.494707 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.494649 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k5zx\"" Jul 02 16:55:08.511803 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511778 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssrc6\" (UniqueName: \"kubernetes.io/projected/ad223535-bdc2-46e6-aa2a-f56fff74dde0-kube-api-access-ssrc6\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.511900 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511817 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.511900 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511850 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n5j8b\" (UniqueName: \"kubernetes.io/projected/fd042de0-a4db-45c2-b7b4-745fa4f7f186-kube-api-access-n5j8b\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.511900 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511875 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad223535-bdc2-46e6-aa2a-f56fff74dde0-config-volume\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.512076 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511921 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad223535-bdc2-46e6-aa2a-f56fff74dde0-tmp-dir\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.512076 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.511966 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd042de0-a4db-45c2-b7b4-745fa4f7f186-cert\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.612501 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612446 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ssrc6\" (UniqueName: \"kubernetes.io/projected/ad223535-bdc2-46e6-aa2a-f56fff74dde0-kube-api-access-ssrc6\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.612501 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612475 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.612501 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612498 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n5j8b\" (UniqueName: \"kubernetes.io/projected/fd042de0-a4db-45c2-b7b4-745fa4f7f186-kube-api-access-n5j8b\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.612655 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad223535-bdc2-46e6-aa2a-f56fff74dde0-config-volume\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.612655 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612563 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad223535-bdc2-46e6-aa2a-f56fff74dde0-tmp-dir\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.612655 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:08.612593 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:08.612655 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:08.612654 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:09.112636267 +0000 UTC m=+37.072985323 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:08.612814 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612595 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd042de0-a4db-45c2-b7b4-745fa4f7f186-cert\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.612963 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.612944 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/ad223535-bdc2-46e6-aa2a-f56fff74dde0-tmp-dir\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.613192 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.613157 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/ad223535-bdc2-46e6-aa2a-f56fff74dde0-config-volume\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.616976 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.616954 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fd042de0-a4db-45c2-b7b4-745fa4f7f186-cert\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.621145 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.621110 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssrc6\" (UniqueName: \"kubernetes.io/projected/ad223535-bdc2-46e6-aa2a-f56fff74dde0-kube-api-access-ssrc6\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:08.621219 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.621187 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n5j8b\" (UniqueName: \"kubernetes.io/projected/fd042de0-a4db-45c2-b7b4-745fa4f7f186-kube-api-access-n5j8b\") pod \"ingress-canary-bzpjp\" (UID: \"fd042de0-a4db-45c2-b7b4-745fa4f7f186\") " pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.717519 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.717497 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-bzpjp" Jul 02 16:55:08.848588 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:08.848561 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-bzpjp"] Jul 02 16:55:08.852171 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:55:08.852100 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd042de0_a4db_45c2_b7b4_745fa4f7f186.slice/crio-68d75f50cbe88483555a0a81ea92a3e44fb00e6408efe2adf3ca785f91d911b1 WatchSource:0}: Error finding container 68d75f50cbe88483555a0a81ea92a3e44fb00e6408efe2adf3ca785f91d911b1: Status 404 returned error can't find the container with id 68d75f50cbe88483555a0a81ea92a3e44fb00e6408efe2adf3ca785f91d911b1 Jul 02 16:55:09.116965 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:09.116936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:09.117177 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:09.117114 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:09.117238 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:09.117197 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:10.117180834 +0000 UTC m=+38.077529890 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:09.730820 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:09.730781 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzpjp" event={"ID":"fd042de0-a4db-45c2-b7b4-745fa4f7f186","Type":"ContainerStarted","Data":"68d75f50cbe88483555a0a81ea92a3e44fb00e6408efe2adf3ca785f91d911b1"} Jul 02 16:55:10.125298 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:10.125221 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:10.125459 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:10.125391 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:10.125504 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:10.125474 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:12.125457533 +0000 UTC m=+40.085806588 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:12.142625 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:12.142415 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:12.143378 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:12.142557 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:12.143378 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:12.142698 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:16.142678416 +0000 UTC m=+44.103027491 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:12.740010 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:12.739967 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="dcba1bb1ef4f46ec1c5dd2e110277443086dbf1069b989f2385ead55cdd8d649" exitCode=0 Jul 02 16:55:12.740192 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:12.740037 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"dcba1bb1ef4f46ec1c5dd2e110277443086dbf1069b989f2385ead55cdd8d649"} Jul 02 16:55:12.741551 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:12.741526 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-bzpjp" event={"ID":"fd042de0-a4db-45c2-b7b4-745fa4f7f186","Type":"ContainerStarted","Data":"4d57389e35eacf232f4a2b92e2be92e5e0f3df52ebc50ddca9225a6e17d8b32d"} Jul 02 16:55:12.774733 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:12.774698 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-bzpjp" podStartSLOduration=1.6862648930000002 podStartE2EDuration="4.774684839s" totalImagesPullingTime="3.088419946s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:12.77371758 +0000 UTC m=+40.734066667" watchObservedRunningTime="2026-07-02 16:55:12.774684839 +0000 UTC m=+40.735033942" Jul 02 16:55:13.747595 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:13.747559 2565 generic.go:356] "Generic (PLEG): container finished" podID="14f5451f-a4c8-4cc2-9457-4e12e2ccb347" containerID="7d4067d0e9e4555c226d8cd885a4bd9ddf0476fd6611a2fd3da3ecff5e76aadf" exitCode=0 Jul 02 16:55:13.748268 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:13.747653 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerDied","Data":"7d4067d0e9e4555c226d8cd885a4bd9ddf0476fd6611a2fd3da3ecff5e76aadf"} Jul 02 16:55:14.753431 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:14.753384 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w29hl" event={"ID":"14f5451f-a4c8-4cc2-9457-4e12e2ccb347","Type":"ContainerStarted","Data":"478dba5edc19814d34978fe211e940850b291c3115ef6d0cc9b327011813c641"} Jul 02 16:55:14.776221 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:14.776175 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w29hl" podStartSLOduration=9.838843465 podStartE2EDuration="41.776160657s" totalImagesPullingTime="31.937317192s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:14.774419571 +0000 UTC m=+42.734768649" watchObservedRunningTime="2026-07-02 16:55:14.776160657 +0000 UTC m=+42.736509734" Jul 02 16:55:16.170323 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:16.170286 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:16.170712 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:16.170418 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:16.170712 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:16.170486 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:24.170472367 +0000 UTC m=+52.130821420 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:24.221111 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:24.221073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:24.221588 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:24.221224 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:24.221588 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:24.221303 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:55:40.221285914 +0000 UTC m=+68.181634967 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:33.728977 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:33.728953 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-bzchs" Jul 02 16:55:38.216075 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.216032 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:38.218659 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.218638 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 02 16:55:38.228475 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.228457 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 02 16:55:38.240473 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.240447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xb7xm\" (UniqueName: \"kubernetes.io/projected/330b7496-741f-49eb-8d70-93c0111a44b2-kube-api-access-xb7xm\") pod \"network-check-target-x2vnt\" (UID: \"330b7496-741f-49eb-8d70-93c0111a44b2\") " pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:38.316546 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.316520 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:38.316647 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.316576 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:55:38.318883 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.318865 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 02 16:55:38.319077 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.319057 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 02 16:55:38.326975 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:38.326955 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:55:38.327073 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:38.327016 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:56:42.327001328 +0000 UTC m=+130.287350385 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : secret "metrics-daemon-secret" not found Jul 02 16:55:38.329276 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.329246 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/b801cb53-4e0c-4a93-8a19-49f2a550187f-original-pull-secret\") pod \"global-pull-secret-syncer-qv2h5\" (UID: \"b801cb53-4e0c-4a93-8a19-49f2a550187f\") " pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:38.504676 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.504616 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-qv2h5" Jul 02 16:55:38.512762 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.512742 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-442wp\"" Jul 02 16:55:38.520803 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.520779 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:38.637986 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.637951 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-qv2h5"] Jul 02 16:55:38.640636 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:55:38.640608 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb801cb53_4e0c_4a93_8a19_49f2a550187f.slice/crio-cd3c10ce6b2c58841d21d51262f3032e64c47fa87cc64a5260e0bc57faaf5bb4 WatchSource:0}: Error finding container cd3c10ce6b2c58841d21d51262f3032e64c47fa87cc64a5260e0bc57faaf5bb4: Status 404 returned error can't find the container with id cd3c10ce6b2c58841d21d51262f3032e64c47fa87cc64a5260e0bc57faaf5bb4 Jul 02 16:55:38.653159 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.653138 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-x2vnt"] Jul 02 16:55:38.656417 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:55:38.656399 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod330b7496_741f_49eb_8d70_93c0111a44b2.slice/crio-b1769eddd0a1332ad27d6f8f983c82e78ac0bc8dd138428c7a28c2199b307f8c WatchSource:0}: Error finding container b1769eddd0a1332ad27d6f8f983c82e78ac0bc8dd138428c7a28c2199b307f8c: Status 404 returned error can't find the container with id b1769eddd0a1332ad27d6f8f983c82e78ac0bc8dd138428c7a28c2199b307f8c Jul 02 16:55:38.811086 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.811021 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x2vnt" event={"ID":"330b7496-741f-49eb-8d70-93c0111a44b2","Type":"ContainerStarted","Data":"b1769eddd0a1332ad27d6f8f983c82e78ac0bc8dd138428c7a28c2199b307f8c"} Jul 02 16:55:38.812033 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:38.811994 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qv2h5" event={"ID":"b801cb53-4e0c-4a93-8a19-49f2a550187f","Type":"ContainerStarted","Data":"cd3c10ce6b2c58841d21d51262f3032e64c47fa87cc64a5260e0bc57faaf5bb4"} Jul 02 16:55:40.229229 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:40.229191 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:55:40.229669 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:40.229369 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:55:40.229669 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:55:40.229448 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:56:12.229425408 +0000 UTC m=+100.189774507 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:55:43.825694 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:43.825660 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-qv2h5" event={"ID":"b801cb53-4e0c-4a93-8a19-49f2a550187f","Type":"ContainerStarted","Data":"494cd096f102e0e93fabf0ad68d786d9bb6731a3fd14683eee78064847db62de"} Jul 02 16:55:43.827145 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:43.827106 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-x2vnt" event={"ID":"330b7496-741f-49eb-8d70-93c0111a44b2","Type":"ContainerStarted","Data":"cc6e7d2b59fd797fec470a5eabc62ecca7e387f333379879b7b3d4ca33a85644"} Jul 02 16:55:43.827263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:43.827251 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:55:43.839394 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:43.839347 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-qv2h5" podStartSLOduration=66.566458207 podStartE2EDuration="1m10.839329812s" totalImagesPullingTime="4.272871605s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:43.838870865 +0000 UTC m=+71.799219941" watchObservedRunningTime="2026-07-02 16:55:43.839329812 +0000 UTC m=+71.799678890" Jul 02 16:55:43.854539 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:43.854500 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-x2vnt" podStartSLOduration=66.601965404 podStartE2EDuration="1m10.854490192s" totalImagesPullingTime="4.252524788s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:55:43.853736964 +0000 UTC m=+71.814086040" watchObservedRunningTime="2026-07-02 16:55:43.854490192 +0000 UTC m=+71.814839268" Jul 02 16:55:57.412495 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.412384 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb"] Jul 02 16:55:57.414801 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.414785 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.417164 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.417134 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 02 16:55:57.417264 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.417134 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jul 02 16:55:57.417264 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.417136 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 02 16:55:57.418436 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.418196 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jul 02 16:55:57.418436 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.418222 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jul 02 16:55:57.418436 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.418202 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jul 02 16:55:57.418436 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.418335 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 02 16:55:57.425638 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.425619 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb"] Jul 02 16:55:57.541976 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.541955 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a6943-602b-45ad-878f-c6948494f29a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.542101 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.541995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.542101 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.542020 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqzdm\" (UniqueName: \"kubernetes.io/projected/c45a6943-602b-45ad-878f-c6948494f29a-kube-api-access-sqzdm\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.542255 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.542112 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.542255 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.542172 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.542255 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.542202 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643403 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643367 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a6943-602b-45ad-878f-c6948494f29a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643495 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643423 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643495 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643449 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqzdm\" (UniqueName: \"kubernetes.io/projected/c45a6943-602b-45ad-878f-c6948494f29a-kube-api-access-sqzdm\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643629 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643609 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643692 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.643692 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.643658 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.644178 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.644114 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"additional-service-ca\" (UniqueName: \"kubernetes.io/configmap/c45a6943-602b-45ad-878f-c6948494f29a-additional-service-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.646526 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.646496 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.646640 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.646582 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-ca\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.646640 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.646597 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-hub\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.646752 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.646670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c45a6943-602b-45ad-878f-c6948494f29a-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.652048 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.652023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqzdm\" (UniqueName: \"kubernetes.io/projected/c45a6943-602b-45ad-878f-c6948494f29a-kube-api-access-sqzdm\") pod \"cluster-proxy-proxy-agent-547d8d6885-7c8sb\" (UID: \"c45a6943-602b-45ad-878f-c6948494f29a\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.740425 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.740401 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:55:57.863303 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:57.863276 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb"] Jul 02 16:55:57.866326 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:55:57.866295 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc45a6943_602b_45ad_878f_c6948494f29a.slice/crio-2d09a49b7a32a6bedcaae5e90b47ac444133c584c67aaf19deb94b68ea0206c1 WatchSource:0}: Error finding container 2d09a49b7a32a6bedcaae5e90b47ac444133c584c67aaf19deb94b68ea0206c1: Status 404 returned error can't find the container with id 2d09a49b7a32a6bedcaae5e90b47ac444133c584c67aaf19deb94b68ea0206c1 Jul 02 16:55:58.864645 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:55:58.864571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerStarted","Data":"2d09a49b7a32a6bedcaae5e90b47ac444133c584c67aaf19deb94b68ea0206c1"} Jul 02 16:56:01.875650 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:01.875619 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerStarted","Data":"7f65ebe2e8ca6f5909dd5b5d58caa828718c0f476e701366ca3575b40b31012f"} Jul 02 16:56:01.876228 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:01.875662 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerStarted","Data":"517635995fb505c0127ac30a6f04fc7ca5a517b2193648cc210b115f104404a3"} Jul 02 16:56:01.876228 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:01.875677 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerStarted","Data":"c2e1530fa607b27ca323e3d2b49123c309f582fde86fe2705a404f298b27c102"} Jul 02 16:56:01.894002 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:01.893950 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" podStartSLOduration=1.44701705 podStartE2EDuration="4.893936167s" totalImagesPullingTime="3.446919117s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:56:01.892520735 +0000 UTC m=+89.852869791" watchObservedRunningTime="2026-07-02 16:56:01.893936167 +0000 UTC m=+89.854285242" Jul 02 16:56:12.233306 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:12.233275 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:56:12.233668 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:56:12.233429 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:56:12.233668 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:56:12.233495 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:57:16.233476288 +0000 UTC m=+164.193825344 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:56:14.832229 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:14.832200 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-x2vnt" Jul 02 16:56:42.335823 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:56:42.335790 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:56:42.336312 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:56:42.335897 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:56:42.336312 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:56:42.335952 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:44.335938833 +0000 UTC m=+252.296287886 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : secret "metrics-daemon-secret" not found Jul 02 16:57:11.409378 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:57:11.409333 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gfmwc" podUID="ad223535-bdc2-46e6-aa2a-f56fff74dde0" Jul 02 16:57:11.515899 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:57:11.515857 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:57:12.072817 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:57:12.072785 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:57:16.250687 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:57:16.250649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:57:16.251140 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:57:16.250795 2565 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 02 16:57:16.251140 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:57:16.250858 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls podName:ad223535-bdc2-46e6-aa2a-f56fff74dde0 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:18.250844072 +0000 UTC m=+286.211193126 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls") pod "dns-default-gfmwc" (UID: "ad223535-bdc2-46e6-aa2a-f56fff74dde0") : secret "dns-default-metrics-tls" not found Jul 02 16:57:24.488307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:57:24.488223 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:58:28.614926 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.614893 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7"] Jul 02 16:58:28.617706 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.617677 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.618938 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.618913 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:58:28.620137 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.620098 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 02 16:58:28.620137 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.620108 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 02 16:58:28.620374 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.620362 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 02 16:58:28.620963 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.620947 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 02 16:58:28.621033 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.620984 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-ljvn4\"" Jul 02 16:58:28.621586 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.621571 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.623915 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.623894 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 02 16:58:28.624158 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.624139 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s9rkj\"" Jul 02 16:58:28.624582 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.624559 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 02 16:58:28.624799 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.624782 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 02 16:58:28.628667 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.628648 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 02 16:58:28.631590 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.631574 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7"] Jul 02 16:58:28.636017 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.635997 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:58:28.718392 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718364 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718392 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718397 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rkb4w\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718506 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcdpx\" (UniqueName: \"kubernetes.io/projected/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-kube-api-access-fcdpx\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718549 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718566 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718580 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.718786 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718637 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.718786 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.718663 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.819662 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819610 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.819662 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819665 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.819825 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819694 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.819825 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819711 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rkb4w\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.819898 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819828 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.819898 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819869 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcdpx\" (UniqueName: \"kubernetes.io/projected/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-kube-api-access-fcdpx\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.819995 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.819902 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.820046 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.820024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.820103 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.820066 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.820183 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.820138 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.820237 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.820193 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.820391 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:28.820306 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 16:58:28.820391 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:28.820325 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b84f5796b-lfsns: secret "image-registry-tls" not found Jul 02 16:58:28.820391 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:28.820383 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls podName:69967f0e-0896-4356-8ff2-3294974500c5 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:29.320362162 +0000 UTC m=+237.280711217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls") pod "image-registry-6b84f5796b-lfsns" (UID: "69967f0e-0896-4356-8ff2-3294974500c5") : secret "image-registry-tls" not found Jul 02 16:58:28.820975 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.820952 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-config\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.821055 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.821039 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.821177 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.821139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.821312 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.821293 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.822395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.822370 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.822476 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.822439 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-serving-cert\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.822476 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.822444 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.828996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.828970 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.828996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.828991 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcdpx\" (UniqueName: \"kubernetes.io/projected/d350d8d6-949d-4f32-9bdc-33f6070cb5c3-kube-api-access-fcdpx\") pod \"kube-storage-version-migrator-operator-79964ff6f9-bs4x7\" (UID: \"d350d8d6-949d-4f32-9bdc-33f6070cb5c3\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:28.829858 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.829836 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rkb4w\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:28.927803 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:28.927773 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" Jul 02 16:58:29.048603 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:29.048575 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7"] Jul 02 16:58:29.051571 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:58:29.051547 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd350d8d6_949d_4f32_9bdc_33f6070cb5c3.slice/crio-06f23eb32fc5171157c3eed72d510fe0836cc570c85803238b1731a755c323e8 WatchSource:0}: Error finding container 06f23eb32fc5171157c3eed72d510fe0836cc570c85803238b1731a755c323e8: Status 404 returned error can't find the container with id 06f23eb32fc5171157c3eed72d510fe0836cc570c85803238b1731a755c323e8 Jul 02 16:58:29.289571 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:29.289485 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" event={"ID":"d350d8d6-949d-4f32-9bdc-33f6070cb5c3","Type":"ContainerStarted","Data":"06f23eb32fc5171157c3eed72d510fe0836cc570c85803238b1731a755c323e8"} Jul 02 16:58:29.324875 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:29.324838 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:29.325014 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:29.324984 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 16:58:29.325014 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:29.324998 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b84f5796b-lfsns: secret "image-registry-tls" not found Jul 02 16:58:29.325091 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:29.325048 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls podName:69967f0e-0896-4356-8ff2-3294974500c5 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:30.32503442 +0000 UTC m=+238.285383473 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls") pod "image-registry-6b84f5796b-lfsns" (UID: "69967f0e-0896-4356-8ff2-3294974500c5") : secret "image-registry-tls" not found Jul 02 16:58:30.333218 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:30.333183 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:30.333596 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:30.333366 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 16:58:30.333596 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:30.333387 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b84f5796b-lfsns: secret "image-registry-tls" not found Jul 02 16:58:30.333596 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:30.333457 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls podName:69967f0e-0896-4356-8ff2-3294974500c5 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:32.333435501 +0000 UTC m=+240.293784556 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls") pod "image-registry-6b84f5796b-lfsns" (UID: "69967f0e-0896-4356-8ff2-3294974500c5") : secret "image-registry-tls" not found Jul 02 16:58:32.300234 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:32.300198 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" event={"ID":"d350d8d6-949d-4f32-9bdc-33f6070cb5c3","Type":"ContainerStarted","Data":"4ac10fdf33d1b1a0e83a38d2e8e41145801f1ee57cbe31075af0eac0aa1fa4ac"} Jul 02 16:58:32.315497 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:32.315448 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" podStartSLOduration=1.972197542 podStartE2EDuration="4.315435303s" totalImagesPullingTime="2.343237761s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:32.315165928 +0000 UTC m=+240.275515004" watchObservedRunningTime="2026-07-02 16:58:32.315435303 +0000 UTC m=+240.275784378" Jul 02 16:58:32.348759 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:32.348727 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:32.348919 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:32.348847 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 16:58:32.348919 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:32.348858 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b84f5796b-lfsns: secret "image-registry-tls" not found Jul 02 16:58:32.348919 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:32.348912 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls podName:69967f0e-0896-4356-8ff2-3294974500c5 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:36.348894312 +0000 UTC m=+244.309243386 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls") pod "image-registry-6b84f5796b-lfsns" (UID: "69967f0e-0896-4356-8ff2-3294974500c5") : secret "image-registry-tls" not found Jul 02 16:58:33.684481 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.684452 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-qflwq"] Jul 02 16:58:33.688866 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.688463 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.691661 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.691637 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 02 16:58:33.691909 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.691886 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 02 16:58:33.691909 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.691905 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 02 16:58:33.692713 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.692691 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-pdlkg\"" Jul 02 16:58:33.692820 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.692757 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 02 16:58:33.696831 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.696808 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qflwq"] Jul 02 16:58:33.760429 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.760390 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.760429 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.760430 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03c864f-be23-47d2-bfbb-4e7113473a11-data-volume\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.760693 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.760466 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b03c864f-be23-47d2-bfbb-4e7113473a11-crio-socket\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.760693 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.760582 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhmg9\" (UniqueName: \"kubernetes.io/projected/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-api-access-jhmg9\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.760693 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.760619 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861526 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861493 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861624 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861530 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03c864f-be23-47d2-bfbb-4e7113473a11-data-volume\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861624 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861561 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b03c864f-be23-47d2-bfbb-4e7113473a11-crio-socket\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861739 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861633 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/b03c864f-be23-47d2-bfbb-4e7113473a11-crio-socket\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861739 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:33.861670 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:33.861834 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861730 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jhmg9\" (UniqueName: \"kubernetes.io/projected/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-api-access-jhmg9\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861834 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:33.861744 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:34.361722592 +0000 UTC m=+242.322071661 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:33.861834 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861782 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861973 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:33.861871 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:33.861973 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.861885 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/b03c864f-be23-47d2-bfbb-4e7113473a11-data-volume\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:33.861973 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:33.861917 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:34.361903817 +0000 UTC m=+242.322252884 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : configmap "kube-rbac-proxy" not found Jul 02 16:58:33.871523 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:33.871493 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhmg9\" (UniqueName: \"kubernetes.io/projected/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-api-access-jhmg9\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:34.366716 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:34.366673 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:34.366974 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:34.366749 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:34.366974 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:34.366814 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:34.366974 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:34.366853 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:34.366974 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:34.366885 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:35.36686485 +0000 UTC m=+243.327213906 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : configmap "kube-rbac-proxy" not found Jul 02 16:58:34.366974 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:34.366903 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:35.366893859 +0000 UTC m=+243.327242914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:35.373214 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:35.373161 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:35.373612 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:35.373243 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:35.373612 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:35.373282 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:35.373612 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:35.373341 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:35.373612 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:35.373344 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:37.373329622 +0000 UTC m=+245.333678676 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : configmap "kube-rbac-proxy" not found Jul 02 16:58:35.373612 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:35.373387 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:37.373375731 +0000 UTC m=+245.333724785 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:36.380013 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:36.379979 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:36.380404 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:36.380147 2565 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 02 16:58:36.380404 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:36.380165 2565 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-6b84f5796b-lfsns: secret "image-registry-tls" not found Jul 02 16:58:36.380404 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:36.380227 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls podName:69967f0e-0896-4356-8ff2-3294974500c5 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:44.380211402 +0000 UTC m=+252.340560469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls") pod "image-registry-6b84f5796b-lfsns" (UID: "69967f0e-0896-4356-8ff2-3294974500c5") : secret "image-registry-tls" not found Jul 02 16:58:36.656811 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:36.656735 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqqbn_25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d/dns-node-resolver/0.log" Jul 02 16:58:37.388283 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:37.388251 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:37.388688 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:37.388316 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:37.388688 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:37.388397 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:37.388688 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:37.388402 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:37.388688 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:37.388462 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:41.388446663 +0000 UTC m=+249.348795717 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:37.388688 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:37.388478 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:41.388469004 +0000 UTC m=+249.348818058 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : configmap "kube-rbac-proxy" not found Jul 02 16:58:37.456905 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:37.456881 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d2hr6_4e80c9d4-a76a-46f4-9554-c258290c37b4/node-ca/0.log" Jul 02 16:58:38.257132 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:38.257093 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzpjp_fd042de0-a4db-45c2-b7b4-745fa4f7f186/serve-healthcheck-canary/0.log" Jul 02 16:58:39.058357 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:39.058335 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-bs4x7_d350d8d6-949d-4f32-9bdc-33f6070cb5c3/kube-storage-version-migrator-operator/0.log" Jul 02 16:58:41.418048 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:41.418015 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:41.418452 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:41.418073 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:41.418452 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:41.418166 2565 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 02 16:58:41.418452 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:41.418169 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:41.418452 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:41.418216 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:49.418202618 +0000 UTC m=+257.378551672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : configmap "kube-rbac-proxy" not found Jul 02 16:58:41.418452 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:41.418235 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:58:49.418222173 +0000 UTC m=+257.378571228 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:44.338801 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.338772 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:58:44.339232 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:44.338933 2565 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 02 16:58:44.339232 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:44.338998 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs podName:863c0cda-2dc4-467b-bf4d-5f8ef80a90a9 nodeName:}" failed. No retries permitted until 2026-07-02 17:00:46.338982179 +0000 UTC m=+374.299331233 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs") pod "network-metrics-daemon-g7wz9" (UID: "863c0cda-2dc4-467b-bf4d-5f8ef80a90a9") : secret "metrics-daemon-secret" not found Jul 02 16:58:44.439412 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.439385 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:44.441828 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.441812 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"image-registry-6b84f5796b-lfsns\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:44.535714 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.535692 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-s9rkj\"" Jul 02 16:58:44.543685 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.543668 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:44.672318 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:44.672290 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:58:44.674960 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:58:44.674934 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod69967f0e_0896_4356_8ff2_3294974500c5.slice/crio-02b229e103e15becad37e6458c8b502491dfb58879978eb0eaec28f37b912f78 WatchSource:0}: Error finding container 02b229e103e15becad37e6458c8b502491dfb58879978eb0eaec28f37b912f78: Status 404 returned error can't find the container with id 02b229e103e15becad37e6458c8b502491dfb58879978eb0eaec28f37b912f78 Jul 02 16:58:45.340542 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:45.340500 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" event={"ID":"69967f0e-0896-4356-8ff2-3294974500c5","Type":"ContainerStarted","Data":"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483"} Jul 02 16:58:45.340946 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:45.340561 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" event={"ID":"69967f0e-0896-4356-8ff2-3294974500c5","Type":"ContainerStarted","Data":"02b229e103e15becad37e6458c8b502491dfb58879978eb0eaec28f37b912f78"} Jul 02 16:58:45.340946 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:45.340597 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:58:45.359597 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:45.359554 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" podStartSLOduration=17.359534533 podStartE2EDuration="17.359534533s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:58:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:58:45.358436915 +0000 UTC m=+253.318786011" watchObservedRunningTime="2026-07-02 16:58:45.359534533 +0000 UTC m=+253.319883609" Jul 02 16:58:49.475659 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:49.475630 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:49.476040 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:49.475682 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:49.476040 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:49.475785 2565 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 02 16:58:49.476040 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:58:49.475861 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls podName:b03c864f-be23-47d2-bfbb-4e7113473a11 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:05.475843996 +0000 UTC m=+273.436193064 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls") pod "insights-runtime-extractor-qflwq" (UID: "b03c864f-be23-47d2-bfbb-4e7113473a11") : secret "insights-runtime-extractor-tls" not found Jul 02 16:58:49.476182 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:49.476107 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/b03c864f-be23-47d2-bfbb-4e7113473a11-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:58:56.869267 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:58:56.869186 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:59:05.493968 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:05.493935 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:59:05.496297 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:05.496279 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/b03c864f-be23-47d2-bfbb-4e7113473a11-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-qflwq\" (UID: \"b03c864f-be23-47d2-bfbb-4e7113473a11\") " pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:59:05.498223 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:05.498206 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-qflwq" Jul 02 16:59:05.625638 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:05.625596 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-qflwq"] Jul 02 16:59:05.628267 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:05.628239 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb03c864f_be23_47d2_bfbb_4e7113473a11.slice/crio-a6b99638779068f902c8bba51781adb857bd3edd7075e95c8ae06f7f98fca0c9 WatchSource:0}: Error finding container a6b99638779068f902c8bba51781adb857bd3edd7075e95c8ae06f7f98fca0c9: Status 404 returned error can't find the container with id a6b99638779068f902c8bba51781adb857bd3edd7075e95c8ae06f7f98fca0c9 Jul 02 16:59:06.403307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:06.403269 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflwq" event={"ID":"b03c864f-be23-47d2-bfbb-4e7113473a11","Type":"ContainerStarted","Data":"a6b99638779068f902c8bba51781adb857bd3edd7075e95c8ae06f7f98fca0c9"} Jul 02 16:59:06.875289 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:06.875236 2565 patch_prober.go:28] interesting pod/image-registry-6b84f5796b-lfsns container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 16:59:06.875684 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:06.875310 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 16:59:09.415888 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:09.415860 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflwq" event={"ID":"b03c864f-be23-47d2-bfbb-4e7113473a11","Type":"ContainerStarted","Data":"55874931e7a220337131e9fccd700a6f2fa4e607acd35a73b47f8ab2acefd70a"} Jul 02 16:59:10.421354 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:10.421327 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflwq" event={"ID":"b03c864f-be23-47d2-bfbb-4e7113473a11","Type":"ContainerStarted","Data":"11d7d19dd0e9b24d6d9825ef285320d10c37b6773bc784dfc643df73d2679e25"} Jul 02 16:59:13.433422 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:13.433387 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-qflwq" event={"ID":"b03c864f-be23-47d2-bfbb-4e7113473a11","Type":"ContainerStarted","Data":"494b7262f81692ed8d51f72a48090493e45b62dffa524537bfcbe9f764ef3acd"} Jul 02 16:59:13.451789 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:13.451742 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-qflwq" podStartSLOduration=33.709221386 podStartE2EDuration="40.451728326s" totalImagesPullingTime="6.74250694s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 16:58:33 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:13.451280711 +0000 UTC m=+281.411629799" watchObservedRunningTime="2026-07-02 16:59:13.451728326 +0000 UTC m=+281.412077402" Jul 02 16:59:15.073697 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:15.073652 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gfmwc" podUID="ad223535-bdc2-46e6-aa2a-f56fff74dde0" Jul 02 16:59:15.439768 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:15.439737 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:16.874416 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:16.874383 2565 patch_prober.go:28] interesting pod/image-registry-6b84f5796b-lfsns container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 02 16:59:16.874834 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:16.874435 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 02 16:59:18.278299 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.278268 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:18.280835 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.280804 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/ad223535-bdc2-46e6-aa2a-f56fff74dde0-metrics-tls\") pod \"dns-default-gfmwc\" (UID: \"ad223535-bdc2-46e6-aa2a-f56fff74dde0\") " pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:18.294013 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.293987 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-k6bx7"] Jul 02 16:59:18.297395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.297379 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.300100 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.300078 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 02 16:59:18.300515 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.300484 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 02 16:59:18.300632 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.300557 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 02 16:59:18.300857 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.300840 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 02 16:59:18.301437 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.301420 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 02 16:59:18.301524 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.301449 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 02 16:59:18.301524 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.301477 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4v478\"" Jul 02 16:59:18.379293 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-root\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379307 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379325 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-sys\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379339 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-wtmp\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379395 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbqnw\" (UniqueName: \"kubernetes.io/projected/64bc2fc3-4c2a-426f-a801-2775c2892d9b-kube-api-access-gbqnw\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379559 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379411 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379559 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379427 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-metrics-client-ca\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379559 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379449 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.379559 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.379482 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-textfile\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.442463 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.442443 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-mmn65\"" Jul 02 16:59:18.451553 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.451530 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:18.480633 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480600 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-textfile\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480738 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480721 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-root\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480809 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480791 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480852 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480827 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-sys\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480895 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480854 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-wtmp\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480895 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480862 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-root\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.480895 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480881 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gbqnw\" (UniqueName: \"kubernetes.io/projected/64bc2fc3-4c2a-426f-a801-2775c2892d9b-kube-api-access-gbqnw\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481036 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480925 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481036 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480942 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-sys\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481036 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480956 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-metrics-client-ca\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481036 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.480987 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-textfile\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481036 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.481021 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-wtmp\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481336 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:18.481066 2565 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 02 16:59:18.481336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.481154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-accelerators-collector-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.481336 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:18.481221 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls podName:64bc2fc3-4c2a-426f-a801-2775c2892d9b nodeName:}" failed. No retries permitted until 2026-07-02 16:59:18.98119912 +0000 UTC m=+286.941548174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls") pod "node-exporter-k6bx7" (UID: "64bc2fc3-4c2a-426f-a801-2775c2892d9b") : secret "node-exporter-tls" not found Jul 02 16:59:18.481464 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.481447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/64bc2fc3-4c2a-426f-a801-2775c2892d9b-metrics-client-ca\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.483293 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.483270 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.492054 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.492032 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbqnw\" (UniqueName: \"kubernetes.io/projected/64bc2fc3-4c2a-426f-a801-2775c2892d9b-kube-api-access-gbqnw\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.573291 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.573223 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gfmwc"] Jul 02 16:59:18.576562 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:18.576539 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad223535_bdc2_46e6_aa2a_f56fff74dde0.slice/crio-5479bb783bb512e11534d3c6dc887a2dc22dc1c5047f4b5ed0dae539f4f1abfe WatchSource:0}: Error finding container 5479bb783bb512e11534d3c6dc887a2dc22dc1c5047f4b5ed0dae539f4f1abfe: Status 404 returned error can't find the container with id 5479bb783bb512e11534d3c6dc887a2dc22dc1c5047f4b5ed0dae539f4f1abfe Jul 02 16:59:18.985310 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.985282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:18.987623 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:18.987601 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/64bc2fc3-4c2a-426f-a801-2775c2892d9b-node-exporter-tls\") pod \"node-exporter-k6bx7\" (UID: \"64bc2fc3-4c2a-426f-a801-2775c2892d9b\") " pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:19.207005 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.206977 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-k6bx7" Jul 02 16:59:19.217484 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:19.217459 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod64bc2fc3_4c2a_426f_a801_2775c2892d9b.slice/crio-b3879c207bbb1e8a3492b56ab9086fb881b8499d5fdcde6d9cfdaf5b892f465e WatchSource:0}: Error finding container b3879c207bbb1e8a3492b56ab9086fb881b8499d5fdcde6d9cfdaf5b892f465e: Status 404 returned error can't find the container with id b3879c207bbb1e8a3492b56ab9086fb881b8499d5fdcde6d9cfdaf5b892f465e Jul 02 16:59:19.296029 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.295958 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 16:59:19.300629 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.300610 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.303387 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303289 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 16:59:19.303387 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303342 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 16:59:19.303387 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303353 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvnhb\"" Jul 02 16:59:19.303603 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303342 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 16:59:19.303659 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303602 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 16:59:19.303712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303674 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 16:59:19.303712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303681 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 16:59:19.303712 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303606 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 16:59:19.303857 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303752 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 16:59:19.304018 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.303990 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 16:59:19.316325 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.316300 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 16:59:19.388650 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388614 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.388880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388658 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.388880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388690 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.388880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388791 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.388880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388818 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.388880 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388867 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388917 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388942 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388956 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.388973 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.389028 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xr256\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.389072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.389240 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.389187 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.455298 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.455250 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6bx7" event={"ID":"64bc2fc3-4c2a-426f-a801-2775c2892d9b","Type":"ContainerStarted","Data":"b3879c207bbb1e8a3492b56ab9086fb881b8499d5fdcde6d9cfdaf5b892f465e"} Jul 02 16:59:19.461281 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.461252 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfmwc" event={"ID":"ad223535-bdc2-46e6-aa2a-f56fff74dde0","Type":"ContainerStarted","Data":"5479bb783bb512e11534d3c6dc887a2dc22dc1c5047f4b5ed0dae539f4f1abfe"} Jul 02 16:59:19.489852 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.489821 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.489971 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.489858 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.489971 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.489875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.489971 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.489897 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.489971 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.489932 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490102 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490171 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490210 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490252 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490234 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490450 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490260 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490450 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490284 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xr256\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490450 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490317 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490597 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490511 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.490975 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.490949 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.491544 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:19.491524 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle podName:f493a404-793b-4e41-a6a3-608efd4db663 nodeName:}" failed. No retries permitted until 2026-07-02 16:59:19.991504149 +0000 UTC m=+287.951853204 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "f493a404-793b-4e41-a6a3-608efd4db663") : configmap references non-existent config key: ca-bundle.crt Jul 02 16:59:19.493352 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.493327 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.493492 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.493468 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.493571 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.493552 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.493752 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.493706 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.494330 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.494309 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.494420 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.494367 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.494568 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.494544 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.494682 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.494589 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.495579 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.495557 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.506788 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.506760 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xr256\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.994913 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.994875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:19.995878 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:19.995853 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:20.211877 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:20.211841 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 16:59:20.466522 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:20.466263 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6bx7" event={"ID":"64bc2fc3-4c2a-426f-a801-2775c2892d9b","Type":"ContainerStarted","Data":"f2d9ea7b6d772944d560ae4a32b796abe8a171e021f7d11af53fa9b68dfad400"} Jul 02 16:59:20.534825 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:20.534778 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 16:59:20.537331 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:20.537285 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf493a404_793b_4e41_a6a3_608efd4db663.slice/crio-0e9f3169fee6fdbb1cc6456dec7d96c502d1140d258147cef17712d7d77cb80c WatchSource:0}: Error finding container 0e9f3169fee6fdbb1cc6456dec7d96c502d1140d258147cef17712d7d77cb80c: Status 404 returned error can't find the container with id 0e9f3169fee6fdbb1cc6456dec7d96c502d1140d258147cef17712d7d77cb80c Jul 02 16:59:21.471678 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.471643 2565 generic.go:356] "Generic (PLEG): container finished" podID="64bc2fc3-4c2a-426f-a801-2775c2892d9b" containerID="f2d9ea7b6d772944d560ae4a32b796abe8a171e021f7d11af53fa9b68dfad400" exitCode=0 Jul 02 16:59:21.472103 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.471744 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6bx7" event={"ID":"64bc2fc3-4c2a-426f-a801-2775c2892d9b","Type":"ContainerDied","Data":"f2d9ea7b6d772944d560ae4a32b796abe8a171e021f7d11af53fa9b68dfad400"} Jul 02 16:59:21.473742 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.473714 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfmwc" event={"ID":"ad223535-bdc2-46e6-aa2a-f56fff74dde0","Type":"ContainerStarted","Data":"f826af7918a71da40ae17f1d0c54ea0546720f4a28376a34e4aaffd72a52878e"} Jul 02 16:59:21.473870 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.473778 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gfmwc" event={"ID":"ad223535-bdc2-46e6-aa2a-f56fff74dde0","Type":"ContainerStarted","Data":"cae7fbeff905a450f1ab950255fe2c4e923481d3950c895c2a08c1328ed384e4"} Jul 02 16:59:21.473870 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.473802 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:21.474927 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.474897 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"0e9f3169fee6fdbb1cc6456dec7d96c502d1140d258147cef17712d7d77cb80c"} Jul 02 16:59:21.507263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.507221 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-gfmwc" podStartSLOduration=251.703058293 podStartE2EDuration="4m13.507206707s" totalImagesPullingTime="1.804148414s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:55:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:21.506225022 +0000 UTC m=+289.466574099" watchObservedRunningTime="2026-07-02 16:59:21.507206707 +0000 UTC m=+289.467555784" Jul 02 16:59:21.888638 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:21.888601 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" containerID="cri-o://e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483" gracePeriod=30 Jul 02 16:59:22.115476 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.115456 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:59:22.211378 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211352 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211378 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211380 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211572 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211407 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rkb4w\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211572 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211429 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211572 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211445 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211572 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211516 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211572 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211570 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.211813 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.211603 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca\") pod \"69967f0e-0896-4356-8ff2-3294974500c5\" (UID: \"69967f0e-0896-4356-8ff2-3294974500c5\") " Jul 02 16:59:22.212372 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.212344 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 16:59:22.213847 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.213788 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 16:59:22.213847 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.213814 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w" (OuterVolumeSpecName: "kube-api-access-rkb4w") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "kube-api-access-rkb4w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 16:59:22.213996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.213864 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 16:59:22.214042 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.214023 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 16:59:22.214186 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.214167 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 16:59:22.214229 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.214202 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 16:59:22.219929 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.219909 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "69967f0e-0896-4356-8ff2-3294974500c5" (UID: "69967f0e-0896-4356-8ff2-3294974500c5"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 16:59:22.312703 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312655 2565 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-image-registry-private-configuration\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312703 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312675 2565 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/69967f0e-0896-4356-8ff2-3294974500c5-installation-pull-secrets\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312703 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312685 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rkb4w\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-kube-api-access-rkb4w\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312703 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312694 2565 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-bound-sa-token\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312703 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312703 2565 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/69967f0e-0896-4356-8ff2-3294974500c5-registry-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312925 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312710 2565 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-registry-certificates\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312925 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312718 2565 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/69967f0e-0896-4356-8ff2-3294974500c5-ca-trust-extracted\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.312925 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.312726 2565 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/69967f0e-0896-4356-8ff2-3294974500c5-trusted-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 16:59:22.478888 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.478857 2565 generic.go:356] "Generic (PLEG): container finished" podID="69967f0e-0896-4356-8ff2-3294974500c5" containerID="e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483" exitCode=0 Jul 02 16:59:22.479263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.478901 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" Jul 02 16:59:22.479263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.478953 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" event={"ID":"69967f0e-0896-4356-8ff2-3294974500c5","Type":"ContainerDied","Data":"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483"} Jul 02 16:59:22.479263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.478996 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6b84f5796b-lfsns" event={"ID":"69967f0e-0896-4356-8ff2-3294974500c5","Type":"ContainerDied","Data":"02b229e103e15becad37e6458c8b502491dfb58879978eb0eaec28f37b912f78"} Jul 02 16:59:22.479263 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.479032 2565 scope.go:117] "RemoveContainer" containerID="e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483" Jul 02 16:59:22.481465 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.481435 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6bx7" event={"ID":"64bc2fc3-4c2a-426f-a801-2775c2892d9b","Type":"ContainerStarted","Data":"3c6d9160a537a7ea8ab4b46be982d3a8ee709935d9b09b25453117ae6e3d41fc"} Jul 02 16:59:22.481566 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.481501 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-k6bx7" event={"ID":"64bc2fc3-4c2a-426f-a801-2775c2892d9b","Type":"ContainerStarted","Data":"0b352521b9fe6b06e1ca750439b082d56dbf814bed81403fd5d4f44b99555b65"} Jul 02 16:59:22.483250 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.483227 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="1839cdb4fef57a83e2d52b34c7c588163d730d4cc9b925b02e7a627adcca0322" exitCode=0 Jul 02 16:59:22.483382 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.483313 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"1839cdb4fef57a83e2d52b34c7c588163d730d4cc9b925b02e7a627adcca0322"} Jul 02 16:59:22.488647 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.488580 2565 scope.go:117] "RemoveContainer" containerID="e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483" Jul 02 16:59:22.490268 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:22.490043 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483\": container with ID starting with e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483 not found: ID does not exist" containerID="e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483" Jul 02 16:59:22.490342 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.490281 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483"} err="failed to get container status \"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483\": rpc error: code = NotFound desc = could not find container \"e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483\": container with ID starting with e7a2bcd1c402f8ffad85928de7b769f71b5fed52fce35c7fedbdaecb4fd54483 not found: ID does not exist" Jul 02 16:59:22.504366 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.504331 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-k6bx7" podStartSLOduration=3.248984083 podStartE2EDuration="4.504320596s" totalImagesPullingTime="1.164688693s" totalInitContainerRuntime="90.64782ms" isStatefulPod=true podCreationTimestamp="2026-07-02 16:59:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:22.503518873 +0000 UTC m=+290.463867962" watchObservedRunningTime="2026-07-02 16:59:22.504320596 +0000 UTC m=+290.464669671" Jul 02 16:59:22.527196 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.527178 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-77fccdd475-sgs6n"] Jul 02 16:59:22.527391 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.527380 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" Jul 02 16:59:22.527431 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.527393 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" Jul 02 16:59:22.527461 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.527431 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="69967f0e-0896-4356-8ff2-3294974500c5" containerName="registry" Jul 02 16:59:22.531868 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.531853 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.534931 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.534910 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-fjj4et486qafa\"" Jul 02 16:59:22.535048 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.535012 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 02 16:59:22.535109 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.535098 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 02 16:59:22.535185 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.535170 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 02 16:59:22.535245 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.535186 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 02 16:59:22.535297 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.535247 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zhq7c\"" Jul 02 16:59:22.542422 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.542404 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77fccdd475-sgs6n"] Jul 02 16:59:22.546381 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.546359 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:59:22.553771 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.553755 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-6b84f5796b-lfsns"] Jul 02 16:59:22.616203 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616153 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-client-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616203 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be009aff-7e76-468c-9094-c0cdfb883fdb-audit-log\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616208 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-metrics-server-audit-profiles\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616268 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616298 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6qtb\" (UniqueName: \"kubernetes.io/projected/be009aff-7e76-468c-9094-c0cdfb883fdb-kube-api-access-w6qtb\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616336 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616318 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-tls\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.616473 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.616340 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-client-certs\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717226 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717195 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-client-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717314 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717241 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be009aff-7e76-468c-9094-c0cdfb883fdb-audit-log\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717314 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717258 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-metrics-server-audit-profiles\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717405 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717372 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717448 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717404 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6qtb\" (UniqueName: \"kubernetes.io/projected/be009aff-7e76-468c-9094-c0cdfb883fdb-kube-api-access-w6qtb\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717448 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717424 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-tls\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717448 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717443 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-client-certs\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.717608 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.717588 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/be009aff-7e76-468c-9094-c0cdfb883fdb-audit-log\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.718146 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.718076 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.718315 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.718297 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/be009aff-7e76-468c-9094-c0cdfb883fdb-metrics-server-audit-profiles\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.719809 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.719779 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-client-ca-bundle\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.719922 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.719904 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-client-certs\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.719958 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.719945 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/be009aff-7e76-468c-9094-c0cdfb883fdb-secret-metrics-server-tls\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.725041 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.725008 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6qtb\" (UniqueName: \"kubernetes.io/projected/be009aff-7e76-468c-9094-c0cdfb883fdb-kube-api-access-w6qtb\") pod \"metrics-server-77fccdd475-sgs6n\" (UID: \"be009aff-7e76-468c-9094-c0cdfb883fdb\") " pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.843084 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.843060 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:22.961937 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:22.961909 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-77fccdd475-sgs6n"] Jul 02 16:59:22.965073 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:22.965047 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe009aff_7e76_468c_9094_c0cdfb883fdb.slice/crio-b4abce2e1e0b6a847468f2ebba29f029a9a6781a51e4df03ab837057a06fcad5 WatchSource:0}: Error finding container b4abce2e1e0b6a847468f2ebba29f029a9a6781a51e4df03ab837057a06fcad5: Status 404 returned error can't find the container with id b4abce2e1e0b6a847468f2ebba29f029a9a6781a51e4df03ab837057a06fcad5 Jul 02 16:59:23.017551 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.017529 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt"] Jul 02 16:59:23.020652 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.020635 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.023612 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.023591 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 02 16:59:23.023710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.023638 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-4hfbk\"" Jul 02 16:59:23.045933 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.045913 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt"] Jul 02 16:59:23.120166 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.120132 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gbnv\" (UniqueName: \"kubernetes.io/projected/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-kube-api-access-5gbnv\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.120257 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.120190 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.220886 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.220853 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gbnv\" (UniqueName: \"kubernetes.io/projected/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-kube-api-access-5gbnv\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.221045 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.220916 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.223751 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.223728 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-monitoring-plugin-cert\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.229284 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.229263 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gbnv\" (UniqueName: \"kubernetes.io/projected/6eaf70fc-f4c3-4b7b-9dbf-766744213ee6-kube-api-access-5gbnv\") pod \"monitoring-plugin-cd8656b8-wxrdt\" (UID: \"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6\") " pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.323628 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.323607 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk"] Jul 02 16:59:23.326763 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.326742 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.329189 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329162 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-g9lbf\"" Jul 02 16:59:23.329307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329213 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 02 16:59:23.329307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329291 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 02 16:59:23.329512 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329489 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 02 16:59:23.329613 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329495 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 02 16:59:23.329613 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.329571 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 02 16:59:23.331479 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.331460 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:23.334723 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.334705 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 02 16:59:23.341355 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.341334 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk"] Jul 02 16:59:23.422647 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422620 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422767 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422653 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-metrics-client-ca\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422767 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422730 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-federate-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422806 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-serving-certs-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422839 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ttp5g\" (UniqueName: \"kubernetes.io/projected/f70886d0-3e01-45fc-a5dc-837fa49e14aa-kube-api-access-ttp5g\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422856 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.422884 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.422877 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.466693 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.466657 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt"] Jul 02 16:59:23.475844 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:23.471464 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6eaf70fc_f4c3_4b7b_9dbf_766744213ee6.slice/crio-66fe85b82c320f1bd5b514c456a636dda99dd014bb2744b0d6823584fd39c0ff WatchSource:0}: Error finding container 66fe85b82c320f1bd5b514c456a636dda99dd014bb2744b0d6823584fd39c0ff: Status 404 returned error can't find the container with id 66fe85b82c320f1bd5b514c456a636dda99dd014bb2744b0d6823584fd39c0ff Jul 02 16:59:23.490190 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.490157 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" event={"ID":"be009aff-7e76-468c-9094-c0cdfb883fdb","Type":"ContainerStarted","Data":"b4abce2e1e0b6a847468f2ebba29f029a9a6781a51e4df03ab837057a06fcad5"} Jul 02 16:59:23.491691 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.491665 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" event={"ID":"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6","Type":"ContainerStarted","Data":"66fe85b82c320f1bd5b514c456a636dda99dd014bb2744b0d6823584fd39c0ff"} Jul 02 16:59:23.523405 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523373 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ttp5g\" (UniqueName: \"kubernetes.io/projected/f70886d0-3e01-45fc-a5dc-837fa49e14aa-kube-api-access-ttp5g\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.523505 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523406 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.523505 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523439 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.523730 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523709 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.523806 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523751 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-metrics-client-ca\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523879 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523950 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-federate-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.523989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-serving-certs-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.524668 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.524739 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-metrics-client-ca\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.525056 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.524831 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f70886d0-3e01-45fc-a5dc-837fa49e14aa-serving-certs-ca-bundle\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.526306 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.526283 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.526913 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.526892 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-secret-telemeter-client\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.527532 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.527505 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-federate-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.528268 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.528251 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/f70886d0-3e01-45fc-a5dc-837fa49e14aa-telemeter-client-tls\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.532063 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.532041 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ttp5g\" (UniqueName: \"kubernetes.io/projected/f70886d0-3e01-45fc-a5dc-837fa49e14aa-kube-api-access-ttp5g\") pod \"telemeter-client-7b9fcd79d8-c5pxk\" (UID: \"f70886d0-3e01-45fc-a5dc-837fa49e14aa\") " pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:23.638286 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:23.638267 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" Jul 02 16:59:24.158722 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.158691 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk"] Jul 02 16:59:24.159850 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:24.159824 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf70886d0_3e01_45fc_a5dc_837fa49e14aa.slice/crio-39fb1244bea3b9762b565c7c942bb2d78d5bfe9c04d7494ed654d337a10dc46f WatchSource:0}: Error finding container 39fb1244bea3b9762b565c7c942bb2d78d5bfe9c04d7494ed654d337a10dc46f: Status 404 returned error can't find the container with id 39fb1244bea3b9762b565c7c942bb2d78d5bfe9c04d7494ed654d337a10dc46f Jul 02 16:59:24.413088 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.413004 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 16:59:24.417257 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.417225 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.420045 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420011 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7nmoiob0pk1gh\"" Jul 02 16:59:24.420045 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420011 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 16:59:24.420250 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420017 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 16:59:24.420320 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420292 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 16:59:24.420364 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420354 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 16:59:24.420418 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420297 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 16:59:24.420588 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420569 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 16:59:24.420669 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420606 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 16:59:24.420853 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420832 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 16:59:24.420947 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420841 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 16:59:24.420947 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.420875 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 16:59:24.421480 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.421280 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 16:59:24.421480 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.421284 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 16:59:24.421480 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.421379 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h8tnx\"" Jul 02 16:59:24.424952 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.424930 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 16:59:24.426937 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.426919 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 16:59:24.432011 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.431993 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 16:59:24.494495 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.494459 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="69967f0e-0896-4356-8ff2-3294974500c5" path="/var/lib/kubelet/pods/69967f0e-0896-4356-8ff2-3294974500c5/volumes" Jul 02 16:59:24.497876 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.497847 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"8dc94bf7c90b0b05a7af5bb8aebec097c70a31e4343d446d5ace6230b17507b3"} Jul 02 16:59:24.497996 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.497912 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"9a8bab472db9efa63c0e32ed751ec64b59b9d7f18d5edddb458e5e2614de3b1d"} Jul 02 16:59:24.499070 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.499045 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" event={"ID":"f70886d0-3e01-45fc-a5dc-837fa49e14aa","Type":"ContainerStarted","Data":"39fb1244bea3b9762b565c7c942bb2d78d5bfe9c04d7494ed654d337a10dc46f"} Jul 02 16:59:24.531273 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531246 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531375 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531284 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531375 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531375 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531335 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531522 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531386 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531522 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531418 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531522 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531434 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531522 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531462 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531528 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531570 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531602 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4rfpq\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531624 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531662 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531679 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.531710 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531694 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.532043 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531722 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.532043 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.532043 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531780 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.532043 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531807 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.532043 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.531837 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632674 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632649 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632791 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632692 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632791 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632718 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632791 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632741 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632791 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632773 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632809 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632832 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632856 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632876 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632934 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632964 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.632992 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.632990 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.633427 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.633024 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.633427 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.633030 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.633427 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.633045 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.633427 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.633070 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.635254 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.635225 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.635953 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.635929 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.636077 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636051 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.636416 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636379 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636809 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636855 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636885 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636912 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4rfpq\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.636939 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.637067 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.637043 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.638813 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.638757 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.638813 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.638778 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.639174 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.639154 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.639584 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.639560 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.639837 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.639815 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.640314 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.640288 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.640485 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.640458 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.640891 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.640691 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.641431 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.641393 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.641852 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.641808 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.642083 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.642047 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.642380 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.642363 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.647037 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.647014 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4rfpq\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq\") pod \"prometheus-k8s-0\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:24.730652 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:24.730522 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:25.181151 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.180958 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 16:59:25.184265 ip-10-0-130-120 kubenswrapper[2565]: W0702 16:59:25.184242 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1b80f6a5_37f1_4c15_a662_4c51bd1b1147.slice/crio-43512f95df39eef129249345c5d0927d21fdc2edb069c93b7404d484334d213b WatchSource:0}: Error finding container 43512f95df39eef129249345c5d0927d21fdc2edb069c93b7404d484334d213b: Status 404 returned error can't find the container with id 43512f95df39eef129249345c5d0927d21fdc2edb069c93b7404d484334d213b Jul 02 16:59:25.505188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.505075 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" event={"ID":"6eaf70fc-f4c3-4b7b-9dbf-766744213ee6","Type":"ContainerStarted","Data":"d87e91c4a4b98b795cffcda610b1d7c50bd2ef0ad3f246b6613d9dcbe05b7968"} Jul 02 16:59:25.505188 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.505152 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:25.509873 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.509817 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"df8a7f9d30e908ff692f35b1a6b4a157c6dac610617e54de9ae3243f8e5f4eba"} Jul 02 16:59:25.510020 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.509919 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"c4aa302d8fa19bce028e00df1c188eedc23013d803260fa670e9803b090e7b33"} Jul 02 16:59:25.510020 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.509962 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"a4722a1f8334c4679763529f938a4a2b7a9b67be4caf5203d699d57e8ec2f17c"} Jul 02 16:59:25.511222 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.511204 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" Jul 02 16:59:25.511964 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.511931 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" event={"ID":"be009aff-7e76-468c-9094-c0cdfb883fdb","Type":"ContainerStarted","Data":"ee272aebcadc99cf070182a6d0bdc6defa39a8f8ab2aad4766bfd67cbe990945"} Jul 02 16:59:25.514219 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.514183 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" exitCode=0 Jul 02 16:59:25.514219 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.514218 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} Jul 02 16:59:25.514382 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.514273 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"43512f95df39eef129249345c5d0927d21fdc2edb069c93b7404d484334d213b"} Jul 02 16:59:25.521307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.521267 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-cd8656b8-wxrdt" podStartSLOduration=1.926809435 podStartE2EDuration="3.521252385s" totalImagesPullingTime="1.59444295s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:25.519806251 +0000 UTC m=+293.480155331" watchObservedRunningTime="2026-07-02 16:59:25.521252385 +0000 UTC m=+293.481601462" Jul 02 16:59:25.535578 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:25.535534 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" podStartSLOduration=1.484816334 podStartE2EDuration="3.53551755s" totalImagesPullingTime="2.050701216s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:22 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:25.534989017 +0000 UTC m=+293.495338123" watchObservedRunningTime="2026-07-02 16:59:25.53551755 +0000 UTC m=+293.495866628" Jul 02 16:59:26.524869 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:26.524812 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerStarted","Data":"bbe6568fff9ec41c68ce8dd0448703ed2ae41c9c69bfa1291183913a8bb5716b"} Jul 02 16:59:26.551835 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:26.551239 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.018644968 podStartE2EDuration="7.551218794s" totalImagesPullingTime="3.51625421s" totalInitContainerRuntime="16.319616ms" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:19 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:26.548790602 +0000 UTC m=+294.509139681" watchObservedRunningTime="2026-07-02 16:59:26.551218794 +0000 UTC m=+294.511567874" Jul 02 16:59:27.488957 ip-10-0-130-120 kubenswrapper[2565]: E0702 16:59:27.488914 2565 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-g7wz9" podUID="863c0cda-2dc4-467b-bf4d-5f8ef80a90a9" Jul 02 16:59:27.531777 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:27.531736 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" event={"ID":"f70886d0-3e01-45fc-a5dc-837fa49e14aa","Type":"ContainerStarted","Data":"4c3545217c5b79fbc84cd026d84c51f8f6455ca6a036b866a69056be3ceb9c0c"} Jul 02 16:59:27.532249 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:27.531846 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" event={"ID":"f70886d0-3e01-45fc-a5dc-837fa49e14aa","Type":"ContainerStarted","Data":"ea5b7a9d5427149e8da56864dfbd49588cf0952abbb790556c266d021d21a93e"} Jul 02 16:59:27.532249 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:27.531880 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" event={"ID":"f70886d0-3e01-45fc-a5dc-837fa49e14aa","Type":"ContainerStarted","Data":"6d158881df73440322d6f9846dccb222b2b83af3cadbcaef812256d59f552b16"} Jul 02 16:59:27.557035 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:27.556980 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7b9fcd79d8-c5pxk" podStartSLOduration=2.146243428 podStartE2EDuration="4.556964406s" totalImagesPullingTime="2.410720978s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:27.555821458 +0000 UTC m=+295.516170535" watchObservedRunningTime="2026-07-02 16:59:27.556964406 +0000 UTC m=+295.517313482" Jul 02 16:59:29.540989 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:29.540956 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} Jul 02 16:59:29.541346 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:29.541017 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} Jul 02 16:59:31.485297 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.485270 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gfmwc" Jul 02 16:59:31.551728 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.551691 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} Jul 02 16:59:31.551862 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.551777 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} Jul 02 16:59:31.551862 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.551822 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} Jul 02 16:59:31.551947 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.551861 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerStarted","Data":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} Jul 02 16:59:31.577289 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:31.577223 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.372048474 podStartE2EDuration="7.57720125s" totalImagesPullingTime="5.181398273s" totalInitContainerRuntime="23.754503ms" isStatefulPod=false podCreationTimestamp="2026-07-02 16:59:24 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 16:59:31.576440281 +0000 UTC m=+299.536789393" watchObservedRunningTime="2026-07-02 16:59:31.57720125 +0000 UTC m=+299.537550379" Jul 02 16:59:32.460143 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:32.460102 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:59:32.460280 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:32.460175 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 16:59:32.462846 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:32.462829 2565 kubelet.go:1648] "Image garbage collection succeeded" Jul 02 16:59:34.730917 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:34.730879 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 16:59:37.742278 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:37.742231 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" podUID="c45a6943-602b-45ad-878f-c6948494f29a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 16:59:42.489902 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:42.489863 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 16:59:42.843687 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:42.843614 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:42.843829 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:42.843695 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 16:59:47.742081 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:47.742038 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" podUID="c45a6943-602b-45ad-878f-c6948494f29a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 16:59:52.623718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:52.623649 2565 generic.go:356] "Generic (PLEG): container finished" podID="d350d8d6-949d-4f32-9bdc-33f6070cb5c3" containerID="4ac10fdf33d1b1a0e83a38d2e8e41145801f1ee57cbe31075af0eac0aa1fa4ac" exitCode=0 Jul 02 16:59:52.623718 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:52.623690 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" event={"ID":"d350d8d6-949d-4f32-9bdc-33f6070cb5c3","Type":"ContainerDied","Data":"4ac10fdf33d1b1a0e83a38d2e8e41145801f1ee57cbe31075af0eac0aa1fa4ac"} Jul 02 16:59:52.624082 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:52.623947 2565 scope.go:117] "RemoveContainer" containerID="4ac10fdf33d1b1a0e83a38d2e8e41145801f1ee57cbe31075af0eac0aa1fa4ac" Jul 02 16:59:53.628932 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:53.628898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-79964ff6f9-bs4x7" event={"ID":"d350d8d6-949d-4f32-9bdc-33f6070cb5c3","Type":"ContainerStarted","Data":"a252f7e477d9e82f12a874bbb4121fa1c9a3cd8c98977a47ef5ac3d5ebf8e321"} Jul 02 16:59:57.741893 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:57.741844 2565 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" podUID="c45a6943-602b-45ad-878f-c6948494f29a" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jul 02 16:59:57.742307 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:57.741934 2565 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" Jul 02 16:59:57.742408 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:57.742389 2565 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"7f65ebe2e8ca6f5909dd5b5d58caa828718c0f476e701366ca3575b40b31012f"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jul 02 16:59:57.742444 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:57.742431 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" podUID="c45a6943-602b-45ad-878f-c6948494f29a" containerName="service-proxy" containerID="cri-o://7f65ebe2e8ca6f5909dd5b5d58caa828718c0f476e701366ca3575b40b31012f" gracePeriod=30 Jul 02 16:59:58.650472 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:58.650439 2565 generic.go:356] "Generic (PLEG): container finished" podID="c45a6943-602b-45ad-878f-c6948494f29a" containerID="7f65ebe2e8ca6f5909dd5b5d58caa828718c0f476e701366ca3575b40b31012f" exitCode=2 Jul 02 16:59:58.650660 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:58.650521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerDied","Data":"7f65ebe2e8ca6f5909dd5b5d58caa828718c0f476e701366ca3575b40b31012f"} Jul 02 16:59:58.650660 ip-10-0-130-120 kubenswrapper[2565]: I0702 16:59:58.650574 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-547d8d6885-7c8sb" event={"ID":"c45a6943-602b-45ad-878f-c6948494f29a","Type":"ContainerStarted","Data":"b30f7bc3f63dbb943481f6ebaeb0b8d81923fa795c77fab0c320696b70e6da95"} Jul 02 17:00:02.849239 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:02.849205 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 17:00:02.853086 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:02.853063 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-77fccdd475-sgs6n" Jul 02 17:00:24.731217 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:24.731133 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:24.750914 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:24.750889 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:24.768489 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:24.768466 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:38.589935 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.589892 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:38.590411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590240 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="alertmanager" containerID="cri-o://9a8bab472db9efa63c0e32ed751ec64b59b9d7f18d5edddb458e5e2614de3b1d" gracePeriod=120 Jul 02 17:00:38.590411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590269 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-metric" containerID="cri-o://df8a7f9d30e908ff692f35b1a6b4a157c6dac610617e54de9ae3243f8e5f4eba" gracePeriod=120 Jul 02 17:00:38.590411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590330 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="prom-label-proxy" containerID="cri-o://bbe6568fff9ec41c68ce8dd0448703ed2ae41c9c69bfa1291183913a8bb5716b" gracePeriod=120 Jul 02 17:00:38.590411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590283 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-web" containerID="cri-o://a4722a1f8334c4679763529f938a4a2b7a9b67be4caf5203d699d57e8ec2f17c" gracePeriod=120 Jul 02 17:00:38.590411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590288 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="config-reloader" containerID="cri-o://8dc94bf7c90b0b05a7af5bb8aebec097c70a31e4343d446d5ace6230b17507b3" gracePeriod=120 Jul 02 17:00:38.590676 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.590330 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy" containerID="cri-o://c4aa302d8fa19bce028e00df1c188eedc23013d803260fa670e9803b090e7b33" gracePeriod=120 Jul 02 17:00:38.806663 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806631 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="bbe6568fff9ec41c68ce8dd0448703ed2ae41c9c69bfa1291183913a8bb5716b" exitCode=0 Jul 02 17:00:38.806663 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806654 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="c4aa302d8fa19bce028e00df1c188eedc23013d803260fa670e9803b090e7b33" exitCode=0 Jul 02 17:00:38.806663 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806662 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="8dc94bf7c90b0b05a7af5bb8aebec097c70a31e4343d446d5ace6230b17507b3" exitCode=0 Jul 02 17:00:38.806663 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806668 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="9a8bab472db9efa63c0e32ed751ec64b59b9d7f18d5edddb458e5e2614de3b1d" exitCode=0 Jul 02 17:00:38.806852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806688 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"bbe6568fff9ec41c68ce8dd0448703ed2ae41c9c69bfa1291183913a8bb5716b"} Jul 02 17:00:38.806852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806738 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"c4aa302d8fa19bce028e00df1c188eedc23013d803260fa670e9803b090e7b33"} Jul 02 17:00:38.806852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806764 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"8dc94bf7c90b0b05a7af5bb8aebec097c70a31e4343d446d5ace6230b17507b3"} Jul 02 17:00:38.806852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:38.806789 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"9a8bab472db9efa63c0e32ed751ec64b59b9d7f18d5edddb458e5e2614de3b1d"} Jul 02 17:00:39.813388 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.813356 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="df8a7f9d30e908ff692f35b1a6b4a157c6dac610617e54de9ae3243f8e5f4eba" exitCode=0 Jul 02 17:00:39.813388 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.813378 2565 generic.go:356] "Generic (PLEG): container finished" podID="f493a404-793b-4e41-a6a3-608efd4db663" containerID="a4722a1f8334c4679763529f938a4a2b7a9b67be4caf5203d699d57e8ec2f17c" exitCode=0 Jul 02 17:00:39.813749 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.813394 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"df8a7f9d30e908ff692f35b1a6b4a157c6dac610617e54de9ae3243f8e5f4eba"} Jul 02 17:00:39.813749 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.813464 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"a4722a1f8334c4679763529f938a4a2b7a9b67be4caf5203d699d57e8ec2f17c"} Jul 02 17:00:39.839537 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.839520 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:39.923028 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923004 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923206 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923041 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923206 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923066 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923206 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923091 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923208 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923239 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923288 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xr256\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923316 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923352 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923619 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923386 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923619 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923413 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923619 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923442 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.923619 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.923480 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca\") pod \"f493a404-793b-4e41-a6a3-608efd4db663\" (UID: \"f493a404-793b-4e41-a6a3-608efd4db663\") " Jul 02 17:00:39.924602 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.924109 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:39.924602 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.924182 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:39.925140 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.925059 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:00:39.926575 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.926545 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:00:39.926834 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.926772 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.926946 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.926873 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256" (OuterVolumeSpecName: "kube-api-access-xr256") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "kube-api-access-xr256". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:00:39.927063 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.927017 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out" (OuterVolumeSpecName: "config-out") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:00:39.927356 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.927333 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.927634 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.927609 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.928032 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.928015 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume" (OuterVolumeSpecName: "config-volume") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.928154 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.928095 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.930964 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.930842 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:39.936539 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:39.936516 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config" (OuterVolumeSpecName: "web-config") pod "f493a404-793b-4e41-a6a3-608efd4db663" (UID: "f493a404-793b-4e41-a6a3-608efd4db663"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:40.024711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024690 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-web-config\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024708 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-tls-assets\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024718 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024727 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-config-out\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024747 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024757 2565 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-cluster-tls-config\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024768 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xr256\" (UniqueName: \"kubernetes.io/projected/f493a404-793b-4e41-a6a3-608efd4db663-kube-api-access-xr256\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024777 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024785 2565 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/f493a404-793b-4e41-a6a3-608efd4db663-alertmanager-main-db\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024794 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024802 2565 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-secret-alertmanager-main-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024810 2565 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/f493a404-793b-4e41-a6a3-608efd4db663-config-volume\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.024827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.024817 2565 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f493a404-793b-4e41-a6a3-608efd4db663-metrics-client-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:40.820680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.820602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"f493a404-793b-4e41-a6a3-608efd4db663","Type":"ContainerDied","Data":"0e9f3169fee6fdbb1cc6456dec7d96c502d1140d258147cef17712d7d77cb80c"} Jul 02 17:00:40.820680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.820671 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:40.821145 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.820705 2565 scope.go:117] "RemoveContainer" containerID="bbe6568fff9ec41c68ce8dd0448703ed2ae41c9c69bfa1291183913a8bb5716b" Jul 02 17:00:40.828439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.828421 2565 scope.go:117] "RemoveContainer" containerID="df8a7f9d30e908ff692f35b1a6b4a157c6dac610617e54de9ae3243f8e5f4eba" Jul 02 17:00:40.834966 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.834948 2565 scope.go:117] "RemoveContainer" containerID="c4aa302d8fa19bce028e00df1c188eedc23013d803260fa670e9803b090e7b33" Jul 02 17:00:40.840895 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.840874 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:40.842274 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.842259 2565 scope.go:117] "RemoveContainer" containerID="a4722a1f8334c4679763529f938a4a2b7a9b67be4caf5203d699d57e8ec2f17c" Jul 02 17:00:40.846548 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.846528 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:40.849264 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.849250 2565 scope.go:117] "RemoveContainer" containerID="8dc94bf7c90b0b05a7af5bb8aebec097c70a31e4343d446d5ace6230b17507b3" Jul 02 17:00:40.855869 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.855852 2565 scope.go:117] "RemoveContainer" containerID="9a8bab472db9efa63c0e32ed751ec64b59b9d7f18d5edddb458e5e2614de3b1d" Jul 02 17:00:40.862616 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.862600 2565 scope.go:117] "RemoveContainer" containerID="1839cdb4fef57a83e2d52b34c7c588163d730d4cc9b925b02e7a627adcca0322" Jul 02 17:00:40.869929 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.869909 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:40.870202 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870187 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-web" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870205 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-web" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870216 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="prom-label-proxy" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870223 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="prom-label-proxy" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870235 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="alertmanager" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870243 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="alertmanager" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870258 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-metric" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870267 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-metric" Jul 02 17:00:40.870279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870277 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="init-config-reloader" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870286 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="init-config-reloader" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870307 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870316 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870328 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="config-reloader" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870336 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="config-reloader" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870399 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="alertmanager" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870410 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870421 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="config-reloader" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870428 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="prom-label-proxy" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870437 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-metric" Jul 02 17:00:40.870646 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.870446 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f493a404-793b-4e41-a6a3-608efd4db663" containerName="kube-rbac-proxy-web" Jul 02 17:00:40.875650 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.875631 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:40.878269 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.878250 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 02 17:00:40.878350 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.878252 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 02 17:00:40.878429 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.878412 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 02 17:00:40.878679 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.878665 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 02 17:00:40.879053 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.878961 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 02 17:00:40.879053 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.879008 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-dvnhb\"" Jul 02 17:00:40.879053 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.879014 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 02 17:00:40.879251 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.879013 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 02 17:00:40.879391 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.879376 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 02 17:00:40.883389 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.883371 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 02 17:00:40.886694 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:40.886672 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:41.031777 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031751 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.031777 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031781 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-out\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.031933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031803 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.031933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031841 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt288\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-kube-api-access-xt288\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.031933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031892 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-web-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.031933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031927 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031950 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.031995 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.032017 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032200 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.032059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032200 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.032087 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.032200 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.032134 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133079 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133018 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133079 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133047 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133079 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133065 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133089 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133132 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-out\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133195 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133219 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xt288\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-kube-api-access-xt288\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133267 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-web-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133296 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133335 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133328 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133726 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133726 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133374 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.133726 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.133495 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.134041 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.134018 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.134343 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.134291 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.136381 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.136254 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-out\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.136705 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.136400 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.136705 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.136577 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.136819 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.136704 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.137072 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.137051 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-web-config\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.137198 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.137139 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-config-volume\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.137263 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.137179 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.137263 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.137250 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.138693 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.138671 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.142009 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.141985 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt288\" (UniqueName: \"kubernetes.io/projected/328c9f3b-e5eb-4713-93b7-69bc1d1597fd-kube-api-access-xt288\") pod \"alertmanager-main-0\" (UID: \"328c9f3b-e5eb-4713-93b7-69bc1d1597fd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.186818 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.186799 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 02 17:00:41.321316 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.321294 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 02 17:00:41.323566 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:00:41.323534 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod328c9f3b_e5eb_4713_93b7_69bc1d1597fd.slice/crio-ef5be4c5383050ba18b0df6ae41306da6008d119e1351c31a63f222987568a1e WatchSource:0}: Error finding container ef5be4c5383050ba18b0df6ae41306da6008d119e1351c31a63f222987568a1e: Status 404 returned error can't find the container with id ef5be4c5383050ba18b0df6ae41306da6008d119e1351c31a63f222987568a1e Jul 02 17:00:41.825248 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.825210 2565 generic.go:356] "Generic (PLEG): container finished" podID="328c9f3b-e5eb-4713-93b7-69bc1d1597fd" containerID="54b2f728d3e6666ac9a3b7ed6e422396617623d667ed9cb74fdbb10aad2df0fb" exitCode=0 Jul 02 17:00:41.825676 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.825295 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerDied","Data":"54b2f728d3e6666ac9a3b7ed6e422396617623d667ed9cb74fdbb10aad2df0fb"} Jul 02 17:00:41.825676 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:41.825374 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"ef5be4c5383050ba18b0df6ae41306da6008d119e1351c31a63f222987568a1e"} Jul 02 17:00:42.497012 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.494076 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f493a404-793b-4e41-a6a3-608efd4db663" path="/var/lib/kubelet/pods/f493a404-793b-4e41-a6a3-608efd4db663/volumes" Jul 02 17:00:42.823818 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.823743 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:42.824043 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824015 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="prometheus" containerID="cri-o://7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" gracePeriod=600 Jul 02 17:00:42.824165 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824025 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy" containerID="cri-o://7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" gracePeriod=600 Jul 02 17:00:42.824165 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824063 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="thanos-sidecar" containerID="cri-o://9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" gracePeriod=600 Jul 02 17:00:42.824306 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824153 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-thanos" containerID="cri-o://d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" gracePeriod=600 Jul 02 17:00:42.824306 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824251 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="config-reloader" containerID="cri-o://e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" gracePeriod=600 Jul 02 17:00:42.824306 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.824259 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-web" containerID="cri-o://3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" gracePeriod=600 Jul 02 17:00:42.836925 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.836894 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"2bccb936cb9cc9b9692667ebcc19746ccecbaf26b2b89f24b0d1b4106e65b2d6"} Jul 02 17:00:42.837246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.836989 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"995a2324c8ec4e34558718585843364d683bcc2a7c95155efc886c89748861bc"} Jul 02 17:00:42.837246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.837035 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"c51f6b68dd7006dbe5d5b5bf6694423778acf9691d4311d132559f1a07efcce0"} Jul 02 17:00:42.837246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.837075 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"2b292fff9f269a0d3dd1f950d94941fb2c43dad40767fc2310b1a109541214b7"} Jul 02 17:00:42.837246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.837140 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"da28b58f08db2dee7d8ee9382d1f70336d5399de24a3cb59719b559370c0ab90"} Jul 02 17:00:42.837246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.837185 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"328c9f3b-e5eb-4713-93b7-69bc1d1597fd","Type":"ContainerStarted","Data":"20b2202f737f6880b0171b500d6806a89980e45807d60e5eb46267c4e84cd95f"} Jul 02 17:00:42.881395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:42.881357 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.866612303 podStartE2EDuration="2.881342485s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.730182ms" isStatefulPod=false podCreationTimestamp="2026-07-02 17:00:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:00:42.879805059 +0000 UTC m=+370.840154137" watchObservedRunningTime="2026-07-02 17:00:42.881342485 +0000 UTC m=+370.841691560" Jul 02 17:00:43.069045 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.069022 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:43.148944 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.148871 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.148944 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.148905 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.148944 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.148939 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.148960 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.148992 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149023 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149048 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149073 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149101 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149150 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4rfpq\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149178 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149205 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149241 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149281 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149320 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149357 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149357 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149431 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149459 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149461 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.149666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149520 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\" (UID: \"1b80f6a5-37f1-4c15-a662-4c51bd1b1147\") " Jul 02 17:00:43.150328 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149831 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.150328 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149966 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-trusted-ca-bundle\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.150328 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.149989 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.150488 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.150363 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.150939 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.150715 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.151246 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.151219 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.151628 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.151576 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:00:43.151725 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.151707 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:00:43.152093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.152027 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.152472 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.152445 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.153577 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.153539 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq" (OuterVolumeSpecName: "kube-api-access-4rfpq") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "kube-api-access-4rfpq". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:00:43.153884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.153854 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.154298 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154262 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out" (OuterVolumeSpecName: "config-out") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:00:43.154380 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154319 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.154483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154456 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config" (OuterVolumeSpecName: "config") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.154584 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154494 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.154584 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154546 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:00:43.154678 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.154616 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.155213 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.155189 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.162543 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.162521 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config" (OuterVolumeSpecName: "web-config") pod "1b80f6a5-37f1-4c15-a662-4c51bd1b1147" (UID: "1b80f6a5-37f1-4c15-a662-4c51bd1b1147"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:00:43.250980 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.250959 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.250980 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.250978 2565 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-tls-assets\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.250987 2565 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-metrics-client-certs\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.250996 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-db\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251004 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251012 2565 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config-out\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251021 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251029 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4rfpq\" (UniqueName: \"kubernetes.io/projected/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-kube-api-access-4rfpq\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251036 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-metrics-client-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251045 2565 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-web-config\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251053 2565 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-thanos-prometheus-http-client-file\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251061 2565 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251068 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251076 2565 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251084 2565 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-kube-rbac-proxy\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251091 2565 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-grpc-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251093 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251098 2565 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-config\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.251556 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.251106 2565 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1b80f6a5-37f1-4c15-a662-4c51bd1b1147-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:00:43.843972 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.843936 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" exitCode=0 Jul 02 17:00:43.843972 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.843965 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" exitCode=0 Jul 02 17:00:43.843972 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.843977 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" exitCode=0 Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.843988 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" exitCode=0 Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.843997 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" exitCode=0 Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844007 2565 generic.go:356] "Generic (PLEG): container finished" podID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" exitCode=0 Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844017 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844027 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844094 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844084 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844185 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844223 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844265 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844304 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} Jul 02 17:00:43.844483 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.844344 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1b80f6a5-37f1-4c15-a662-4c51bd1b1147","Type":"ContainerDied","Data":"43512f95df39eef129249345c5d0927d21fdc2edb069c93b7404d484334d213b"} Jul 02 17:00:43.852906 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.852892 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.859615 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.859600 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.866675 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.866654 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.867100 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.867078 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:43.871165 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.871140 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:43.873422 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.873406 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.880052 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.880034 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.886870 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.886852 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.893349 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.893334 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.893624 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.893606 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.893673 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.893632 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.893673 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.893648 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.893918 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.893902 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.893955 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.893924 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.893955 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.893938 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.894183 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.894165 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.894235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894193 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.894235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894207 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.894398 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.894385 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.894433 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894399 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.894433 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894411 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.894596 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.894579 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.894662 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894604 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.894662 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894624 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.894829 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.894814 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.894870 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894832 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.894870 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.894845 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.895072 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:00:43.895054 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.895150 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895079 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.895150 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895097 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.895339 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895318 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.895389 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895340 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.895547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895532 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.895591 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895547 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.895710 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895692 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.895777 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895711 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.895898 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895882 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.895963 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.895899 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.896076 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896061 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.896146 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896076 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.896291 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896275 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.896334 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896291 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.896509 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896492 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.896590 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896511 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.896743 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896728 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.896788 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896745 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.896951 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896934 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.897006 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.896951 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.897152 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897133 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.897233 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897156 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.897337 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897320 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.897388 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897339 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.897508 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897492 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.897546 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897509 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.897680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897663 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.897739 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897681 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.897890 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897875 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.897934 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.897890 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.898059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898042 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.898107 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898067 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.898321 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898304 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.898387 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898322 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.898545 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898510 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.898590 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898547 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.898743 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898721 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.898807 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898745 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.898987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898970 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.899034 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.898989 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.899286 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899242 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.899286 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899286 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.899615 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899590 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.899710 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899616 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.899903 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899885 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.899980 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.899904 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.900198 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900179 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.900198 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900198 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.900377 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900266 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:43.900437 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900420 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.900478 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900437 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.900514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900484 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="thanos-sidecar" Jul 02 17:00:43.900514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900493 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="thanos-sidecar" Jul 02 17:00:43.900514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900502 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="prometheus" Jul 02 17:00:43.900514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900508 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="prometheus" Jul 02 17:00:43.900514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900513 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900519 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900530 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-thanos" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900534 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-thanos" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900545 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="config-reloader" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900552 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="config-reloader" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900564 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="init-config-reloader" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900570 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="init-config-reloader" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900575 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-web" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900581 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-web" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900625 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="config-reloader" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900640 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900646 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-thanos" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900646 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.900687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900662 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900652 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="kube-rbac-proxy-web" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900714 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="thanos-sidecar" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900722 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" containerName="prometheus" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900906 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.900922 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901167 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.901190 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901182 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.901449 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901417 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.901449 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901431 2565 scope.go:117] "RemoveContainer" containerID="d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a" Jul 02 17:00:43.901640 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901624 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a"} err="failed to get container status \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": rpc error: code = NotFound desc = could not find container \"d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a\": container with ID starting with d97fefa1a5eca6e0755fc4f3a47076fe701e1120bca38738fb5daf26ff14e34a not found: ID does not exist" Jul 02 17:00:43.901687 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901640 2565 scope.go:117] "RemoveContainer" containerID="7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57" Jul 02 17:00:43.901853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901835 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57"} err="failed to get container status \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": rpc error: code = NotFound desc = could not find container \"7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57\": container with ID starting with 7f47ada1d518968d73092429cf5e217f149ac396ae3dce3cf1e7f26e1c040a57 not found: ID does not exist" Jul 02 17:00:43.901919 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.901854 2565 scope.go:117] "RemoveContainer" containerID="3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67" Jul 02 17:00:43.902097 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902082 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67"} err="failed to get container status \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": rpc error: code = NotFound desc = could not find container \"3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67\": container with ID starting with 3fb34482e6a23cf5684161625a6735b6fce5afc1571edfdc9477bbc011f5ef67 not found: ID does not exist" Jul 02 17:00:43.902176 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902097 2565 scope.go:117] "RemoveContainer" containerID="9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d" Jul 02 17:00:43.902294 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902279 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d"} err="failed to get container status \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": rpc error: code = NotFound desc = could not find container \"9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d\": container with ID starting with 9ca4115e2751b0a34f575db419c0b0aec4e4510f34354849ac0faff546d5ad6d not found: ID does not exist" Jul 02 17:00:43.902332 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902294 2565 scope.go:117] "RemoveContainer" containerID="e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b" Jul 02 17:00:43.902482 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902467 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b"} err="failed to get container status \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": rpc error: code = NotFound desc = could not find container \"e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b\": container with ID starting with e8b49aa29119528bc9fc7e3276a3f258a89bbedcd09bdd2e61c4ee74df1f445b not found: ID does not exist" Jul 02 17:00:43.902519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902481 2565 scope.go:117] "RemoveContainer" containerID="7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3" Jul 02 17:00:43.902660 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902646 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3"} err="failed to get container status \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": rpc error: code = NotFound desc = could not find container \"7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3\": container with ID starting with 7f2089a9e21046776e4fdcd3c3d216a5465057c9170ee65a465909835a39e3c3 not found: ID does not exist" Jul 02 17:00:43.902708 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902660 2565 scope.go:117] "RemoveContainer" containerID="8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16" Jul 02 17:00:43.902848 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.902825 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16"} err="failed to get container status \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": rpc error: code = NotFound desc = could not find container \"8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16\": container with ID starting with 8c93ce7ab513fcc3a851f5f031238b8a4f1c393d73ca2fc5fdab816c8d5e8f16 not found: ID does not exist" Jul 02 17:00:43.905818 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.905802 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:43.908284 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908264 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 02 17:00:43.908372 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908352 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 02 17:00:43.908430 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908374 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 02 17:00:43.908500 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908484 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 02 17:00:43.908587 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908571 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-7nmoiob0pk1gh\"" Jul 02 17:00:43.908666 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908640 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 02 17:00:43.908742 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908726 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 02 17:00:43.908811 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908798 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 02 17:00:43.908874 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908858 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-h8tnx\"" Jul 02 17:00:43.909014 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.908997 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 02 17:00:43.909080 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.909040 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 02 17:00:43.909149 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.909096 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 02 17:00:43.909200 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.909168 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 02 17:00:43.910431 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.910413 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 02 17:00:43.913162 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.913145 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 02 17:00:43.920303 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.920283 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 02 17:00:43.951171 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:43.951112 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:44.058046 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058011 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058058 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058091 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058147 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058166 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058189 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058204 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058272 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058305 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058322 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058358 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058403 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2ksx2\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-kube-api-access-2ksx2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058460 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058453 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058482 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058513 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058532 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058567 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.058695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.058584 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159225 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159197 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159225 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159417 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159240 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159417 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159254 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159541 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159521 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159594 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159557 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159594 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159578 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159696 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159601 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159696 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159635 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159696 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159657 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159696 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159686 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2ksx2\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-kube-api-access-2ksx2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159713 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159742 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159783 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159851 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.159881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159875 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.160368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159905 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.160368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159948 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.160368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159981 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.160368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.159989 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.160368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.160147 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.161261 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162104 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162457 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162631 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162643 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162720 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.162818 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/66f551bc-f528-4f10-aea5-8af4e1bbebd7-config-out\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.163027 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.163073 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163799 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.163176 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.163799 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.163373 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.165288 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.165239 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.165419 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.165303 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.165604 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.165581 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.165698 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.165683 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-web-config\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.165836 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.165811 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/66f551bc-f528-4f10-aea5-8af4e1bbebd7-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.166410 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.166393 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/66f551bc-f528-4f10-aea5-8af4e1bbebd7-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.170822 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.170803 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2ksx2\" (UniqueName: \"kubernetes.io/projected/66f551bc-f528-4f10-aea5-8af4e1bbebd7-kube-api-access-2ksx2\") pod \"prometheus-k8s-0\" (UID: \"66f551bc-f528-4f10-aea5-8af4e1bbebd7\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.215500 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.215446 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:00:44.349835 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.349805 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 02 17:00:44.353369 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:00:44.353345 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod66f551bc_f528_4f10_aea5_8af4e1bbebd7.slice/crio-d00af3bd1f93e74e59afa6c0263b19a5c6b9a6bf540c128dc8165be2bc60f189 WatchSource:0}: Error finding container d00af3bd1f93e74e59afa6c0263b19a5c6b9a6bf540c128dc8165be2bc60f189: Status 404 returned error can't find the container with id d00af3bd1f93e74e59afa6c0263b19a5c6b9a6bf540c128dc8165be2bc60f189 Jul 02 17:00:44.495183 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.495101 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1b80f6a5-37f1-4c15-a662-4c51bd1b1147" path="/var/lib/kubelet/pods/1b80f6a5-37f1-4c15-a662-4c51bd1b1147/volumes" Jul 02 17:00:44.850527 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.850455 2565 generic.go:356] "Generic (PLEG): container finished" podID="66f551bc-f528-4f10-aea5-8af4e1bbebd7" containerID="d2a97c200b23360b49654c0741ee60db1b35a3f4f30ffc14650a4331b6b131a7" exitCode=0 Jul 02 17:00:44.850527 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.850487 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerDied","Data":"d2a97c200b23360b49654c0741ee60db1b35a3f4f30ffc14650a4331b6b131a7"} Jul 02 17:00:44.850880 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:44.850532 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"d00af3bd1f93e74e59afa6c0263b19a5c6b9a6bf540c128dc8165be2bc60f189"} Jul 02 17:00:45.858592 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858556 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"40d691b50d046271d0850b4110dc661fe5cea5c477ad1752ac59e9afd77154cc"} Jul 02 17:00:45.858948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858617 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"958bff548bcb690c71a4a6fc7d0151bc084b5ce7152a9d1ffc5d5f142cae2286"} Jul 02 17:00:45.858948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858644 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"4a3bb76ebd3720c9831215106675906f6b6c62f4813af203eb8fb45c986ae697"} Jul 02 17:00:45.858948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858671 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"431042c91e8b1be8850b2376d6d5cb42a03d4c94f04591284a30e34b6701820f"} Jul 02 17:00:45.858948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"71fee81ba9e63c04ea6753d95499bf2664306ba093853c0e3c5cff3b2dd9c499"} Jul 02 17:00:45.858948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.858722 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"66f551bc-f528-4f10-aea5-8af4e1bbebd7","Type":"ContainerStarted","Data":"71cb4463864aa6f7c3a43b2d7235017838df8670bf95256869965b3155a5fd34"} Jul 02 17:00:45.887547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:45.887492 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=2.8687021010000002 podStartE2EDuration="2.887479388s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.777287ms" isStatefulPod=false podCreationTimestamp="2026-07-02 17:00:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:00:45.885213661 +0000 UTC m=+373.845562734" watchObservedRunningTime="2026-07-02 17:00:45.887479388 +0000 UTC m=+373.847828462" Jul 02 17:00:46.379983 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.379951 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 17:00:46.382621 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.382602 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/863c0cda-2dc4-467b-bf4d-5f8ef80a90a9-metrics-certs\") pod \"network-metrics-daemon-g7wz9\" (UID: \"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9\") " pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 17:00:46.392855 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.392833 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-6k5zx\"" Jul 02 17:00:46.401655 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.401638 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-g7wz9" Jul 02 17:00:46.528736 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.528714 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-g7wz9"] Jul 02 17:00:46.530965 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:00:46.530938 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod863c0cda_2dc4_467b_bf4d_5f8ef80a90a9.slice/crio-70ccd1d55dcb1442289d9492e53fbd1d51122a72ab7b5c4b097fc082d4f1867d WatchSource:0}: Error finding container 70ccd1d55dcb1442289d9492e53fbd1d51122a72ab7b5c4b097fc082d4f1867d: Status 404 returned error can't find the container with id 70ccd1d55dcb1442289d9492e53fbd1d51122a72ab7b5c4b097fc082d4f1867d Jul 02 17:00:46.532873 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.532852 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:00:46.862958 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:46.862923 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wz9" event={"ID":"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9","Type":"ContainerStarted","Data":"70ccd1d55dcb1442289d9492e53fbd1d51122a72ab7b5c4b097fc082d4f1867d"} Jul 02 17:00:48.874667 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:48.874637 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wz9" event={"ID":"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9","Type":"ContainerStarted","Data":"8c2a65207f67d89b50c3f7c033f28df062641aefc86fea4f97b54eb688ed885e"} Jul 02 17:00:48.875048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:48.874684 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-g7wz9" event={"ID":"863c0cda-2dc4-467b-bf4d-5f8ef80a90a9","Type":"ContainerStarted","Data":"efe5359618b129734ad9068a249ef611840b1ae4137da13ee6a4712146146431"} Jul 02 17:00:48.889521 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:48.889409 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-g7wz9" podStartSLOduration=374.569789957 podStartE2EDuration="6m15.88939093s" totalImagesPullingTime="1.319600973s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 16:54:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:00:48.888567012 +0000 UTC m=+376.848916086" watchObservedRunningTime="2026-07-02 17:00:48.88939093 +0000 UTC m=+376.849740007" Jul 02 17:00:49.215676 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:00:49.215645 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:01:44.216649 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:01:44.216612 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:01:44.232368 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:01:44.232344 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:01:45.101988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:01:45.101963 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 02 17:02:57.791084 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.791053 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b"] Jul 02 17:02:57.794235 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.794220 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.800890 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.800869 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 02 17:02:57.801010 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.800957 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 02 17:02:57.802062 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.802041 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 02 17:02:57.802167 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.802080 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 02 17:02:57.802167 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.802097 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-ss8bb\"" Jul 02 17:02:57.807525 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.807508 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 02 17:02:57.813160 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.813138 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b"] Jul 02 17:02:57.879196 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.879176 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.879282 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.879224 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc964fe-c479-4193-94d7-7e35cf11db6d-manager-config\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.879282 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.879241 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-metrics-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.879352 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.879313 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g4xcc\" (UniqueName: \"kubernetes.io/projected/7dc964fe-c479-4193-94d7-7e35cf11db6d-kube-api-access-g4xcc\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.980617 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.980593 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g4xcc\" (UniqueName: \"kubernetes.io/projected/7dc964fe-c479-4193-94d7-7e35cf11db6d-kube-api-access-g4xcc\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.980716 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.980661 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.980785 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.980738 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc964fe-c479-4193-94d7-7e35cf11db6d-manager-config\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.980785 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.980763 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-metrics-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.981474 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.981454 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/7dc964fe-c479-4193-94d7-7e35cf11db6d-manager-config\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.983186 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.983167 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-metrics-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.983270 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.983197 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7dc964fe-c479-4193-94d7-7e35cf11db6d-cert\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:57.991470 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:57.991447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g4xcc\" (UniqueName: \"kubernetes.io/projected/7dc964fe-c479-4193-94d7-7e35cf11db6d-kube-api-access-g4xcc\") pod \"lws-controller-manager-f87dff94c-qv99b\" (UID: \"7dc964fe-c479-4193-94d7-7e35cf11db6d\") " pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:58.103438 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:58.103386 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:02:58.258819 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:58.258801 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b"] Jul 02 17:02:58.260917 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:02:58.260890 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dc964fe_c479_4193_94d7_7e35cf11db6d.slice/crio-7af0e67a502f5212c9faef0e49827ce565a9d4df6c77b03f94f30753cf652ce1 WatchSource:0}: Error finding container 7af0e67a502f5212c9faef0e49827ce565a9d4df6c77b03f94f30753cf652ce1: Status 404 returned error can't find the container with id 7af0e67a502f5212c9faef0e49827ce565a9d4df6c77b03f94f30753cf652ce1 Jul 02 17:02:58.358428 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:58.358373 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" event={"ID":"7dc964fe-c479-4193-94d7-7e35cf11db6d","Type":"ContainerStarted","Data":"7af0e67a502f5212c9faef0e49827ce565a9d4df6c77b03f94f30753cf652ce1"} Jul 02 17:02:59.151334 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.151301 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl"] Jul 02 17:02:59.156038 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.156018 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.158544 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.158520 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 02 17:02:59.158677 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.158660 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 02 17:02:59.158786 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.158675 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 02 17:02:59.158786 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.158704 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-k2nc7\"" Jul 02 17:02:59.158786 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.158739 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 02 17:02:59.178262 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.178243 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl"] Jul 02 17:02:59.292267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.292242 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.292440 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.292291 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.292440 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.292381 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.292440 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.292420 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.292545 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.292442 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5hjxk\" (UniqueName: \"kubernetes.io/projected/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-kube-api-access-5hjxk\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393207 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393371 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393239 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393371 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393371 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393322 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393371 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393347 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5hjxk\" (UniqueName: \"kubernetes.io/projected/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-kube-api-access-5hjxk\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.393652 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.393620 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-manifests\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.395829 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.395799 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-tmp\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.396205 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.396182 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-webhook-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.396296 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.396267 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-apiservice-cert\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.402234 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.402110 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5hjxk\" (UniqueName: \"kubernetes.io/projected/901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7-kube-api-access-5hjxk\") pod \"opendatahub-operator-controller-manager-845f459c56-77zcl\" (UID: \"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7\") " pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.466294 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.466218 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:02:59.632297 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:02:59.632161 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl"] Jul 02 17:02:59.635759 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:02:59.635733 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod901607d1_a9fb_4d5e_82a4_55e2ec5bc3b7.slice/crio-9996ad382ee387ddb4cc08ac490253569c6555ad2719e63f40661e2608c12fa6 WatchSource:0}: Error finding container 9996ad382ee387ddb4cc08ac490253569c6555ad2719e63f40661e2608c12fa6: Status 404 returned error can't find the container with id 9996ad382ee387ddb4cc08ac490253569c6555ad2719e63f40661e2608c12fa6 Jul 02 17:03:00.370412 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:00.370375 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" event={"ID":"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7","Type":"ContainerStarted","Data":"9996ad382ee387ddb4cc08ac490253569c6555ad2719e63f40661e2608c12fa6"} Jul 02 17:03:01.378251 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:01.378164 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" event={"ID":"7dc964fe-c479-4193-94d7-7e35cf11db6d","Type":"ContainerStarted","Data":"9c51e55dd92a64bec8f77219763de83842963437a833c40e30826d097e8454eb"} Jul 02 17:03:01.378699 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:01.378379 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:03:01.395793 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:01.395722 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" podStartSLOduration=1.6565811990000001 podStartE2EDuration="4.395703924s" totalImagesPullingTime="2.739122725s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:02:57 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:01.394057178 +0000 UTC m=+509.354406255" watchObservedRunningTime="2026-07-02 17:03:01.395703924 +0000 UTC m=+509.356053002" Jul 02 17:03:04.391986 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:04.391954 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" event={"ID":"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7","Type":"ContainerStarted","Data":"1d35756ca44af555b6dd360118d9c4cbb224a2dfb91a33db42b381f6ea1dfeca"} Jul 02 17:03:05.397303 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:05.397270 2565 generic.go:356] "Generic (PLEG): container finished" podID="901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7" containerID="1d35756ca44af555b6dd360118d9c4cbb224a2dfb91a33db42b381f6ea1dfeca" exitCode=0 Jul 02 17:03:05.397781 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:05.397329 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" event={"ID":"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7","Type":"ContainerDied","Data":"1d35756ca44af555b6dd360118d9c4cbb224a2dfb91a33db42b381f6ea1dfeca"} Jul 02 17:03:06.402578 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:06.402546 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" event={"ID":"901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7","Type":"ContainerStarted","Data":"8d8a3aff895d95c2566268acdf945343807af971814703b3c4dd8adbc34bba51"} Jul 02 17:03:06.403146 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:06.402679 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:03:06.424258 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:06.424218 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" podStartSLOduration=2.215811945 podStartE2EDuration="7.424206718s" totalImagesPullingTime="4.173284701s" totalInitContainerRuntime="1.035110072s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:02:59 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:06.422397253 +0000 UTC m=+514.382746343" watchObservedRunningTime="2026-07-02 17:03:06.424206718 +0000 UTC m=+514.384555793" Jul 02 17:03:12.385105 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:12.385070 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-f87dff94c-qv99b" Jul 02 17:03:17.408884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:17.408849 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-845f459c56-77zcl" Jul 02 17:03:27.502884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.502797 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q"] Jul 02 17:03:27.505805 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.505789 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.508255 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508230 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 02 17:03:27.508551 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508526 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 02 17:03:27.508664 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508602 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 02 17:03:27.508728 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508696 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 02 17:03:27.508728 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508705 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 02 17:03:27.508842 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508741 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 02 17:03:27.508842 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.508745 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-fc9hw\"" Jul 02 17:03:27.521286 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.521260 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q"] Jul 02 17:03:27.523012 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.522989 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523153 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523046 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523153 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523072 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdzbd\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-kube-api-access-jdzbd\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523153 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523113 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523298 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523298 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.523298 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.523252 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8736968b-ea34-41f0-842c-8eb181467684-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.623804 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623762 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.623804 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623802 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623822 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8736968b-ea34-41f0-842c-8eb181467684-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623862 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623890 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623904 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdzbd\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-kube-api-access-jdzbd\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.623927 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.624752 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.624529 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.626342 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.626318 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/8736968b-ea34-41f0-842c-8eb181467684-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.626507 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.626493 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.626549 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.626520 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.626630 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.626610 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/8736968b-ea34-41f0-842c-8eb181467684-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.632677 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.632649 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdzbd\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-kube-api-access-jdzbd\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.632813 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.632793 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/8736968b-ea34-41f0-842c-8eb181467684-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-fzw2q\" (UID: \"8736968b-ea34-41f0-842c-8eb181467684\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.814750 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.814668 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:27.952343 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:27.952318 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q"] Jul 02 17:03:27.954995 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:03:27.954966 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8736968b_ea34_41f0_842c_8eb181467684.slice/crio-06a8598f74e99c145a603f9c7b8c7ed12a468475171056646740462c627e0cf0 WatchSource:0}: Error finding container 06a8598f74e99c145a603f9c7b8c7ed12a468475171056646740462c627e0cf0: Status 404 returned error can't find the container with id 06a8598f74e99c145a603f9c7b8c7ed12a468475171056646740462c627e0cf0 Jul 02 17:03:28.495301 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:28.495269 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" event={"ID":"8736968b-ea34-41f0-842c-8eb181467684","Type":"ContainerStarted","Data":"06a8598f74e99c145a603f9c7b8c7ed12a468475171056646740462c627e0cf0"} Jul 02 17:03:31.006098 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:31.006058 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 17:03:31.006365 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:31.006145 2565 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"30892168Ki","pods":"250"} Jul 02 17:03:31.507328 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:31.507294 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" event={"ID":"8736968b-ea34-41f0-842c-8eb181467684","Type":"ContainerStarted","Data":"79e7886d7fe3846ff71c51e0aa5fa09d265c930a4378f6143760fc9094621f98"} Jul 02 17:03:31.507512 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:31.507397 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:03:31.546479 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:31.546417 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" podStartSLOduration=1.497664107 podStartE2EDuration="4.546396626s" totalImagesPullingTime="3.048732519s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:03:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:03:31.544576364 +0000 UTC m=+539.504925434" watchObservedRunningTime="2026-07-02 17:03:31.546396626 +0000 UTC m=+539.506745704" Jul 02 17:03:32.512944 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:03:32.512916 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-fzw2q" Jul 02 17:04:32.485571 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:04:32.485545 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:04:32.486588 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:04:32.486546 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:05:25.675977 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.675931 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2"] Jul 02 17:05:25.679046 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.679031 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.681622 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.681596 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-6frrn\"" Jul 02 17:05:25.681753 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.681624 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 17:05:25.681753 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.681643 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 17:05:25.690546 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.690526 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2"] Jul 02 17:05:25.758081 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.758059 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.758206 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.758103 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5gqvv\" (UniqueName: \"kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.858669 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.858640 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.858755 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.858685 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5gqvv\" (UniqueName: \"kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.859062 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.859045 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.867612 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.867592 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5gqvv\" (UniqueName: \"kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:25.989334 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:25.989280 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:26.121494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.121466 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2"] Jul 02 17:05:26.124083 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:05:26.124054 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf748cf95_fa8d_4919_afee_f79f23fdd134.slice/crio-dca79bc86e0c8bf0b30f85674123cc2c97937458042da4c2c324af18cc5e4ca7 WatchSource:0}: Error finding container dca79bc86e0c8bf0b30f85674123cc2c97937458042da4c2c324af18cc5e4ca7: Status 404 returned error can't find the container with id dca79bc86e0c8bf0b30f85674123cc2c97937458042da4c2c324af18cc5e4ca7 Jul 02 17:05:26.518032 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.518001 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2"] Jul 02 17:05:26.522402 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.522377 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2"] Jul 02 17:05:26.542155 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.542108 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:26.545135 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.545098 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.559947 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.559919 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:26.564758 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.564729 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.564884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.564777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6kn4l\" (UniqueName: \"kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.666339 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.666310 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.666523 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.666352 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6kn4l\" (UniqueName: \"kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.666668 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.666646 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.683516 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.683486 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6kn4l\" (UniqueName: \"kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-44ws8\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:26.857007 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:26.856932 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:27.363557 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:27.363526 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:27.370238 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:05:27.370207 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28b54bdb_f582_4880_84ed_9b1c608b737e.slice/crio-f6255ad7d85977b701636a5a85b6ed10a64de9c7bdb16c79eed3e7e7431ae863 WatchSource:0}: Error finding container f6255ad7d85977b701636a5a85b6ed10a64de9c7bdb16c79eed3e7e7431ae863: Status 404 returned error can't find the container with id f6255ad7d85977b701636a5a85b6ed10a64de9c7bdb16c79eed3e7e7431ae863 Jul 02 17:05:27.989501 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:27.989463 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" event={"ID":"28b54bdb-f582-4880-84ed-9b1c608b737e","Type":"ContainerStarted","Data":"f6255ad7d85977b701636a5a85b6ed10a64de9c7bdb16c79eed3e7e7431ae863"} Jul 02 17:05:31.004405 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.004361 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" containerName="manager" containerID="cri-o://e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0" gracePeriod=2 Jul 02 17:05:31.005990 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.005957 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" event={"ID":"28b54bdb-f582-4880-84ed-9b1c608b737e","Type":"ContainerStarted","Data":"9db9dd0bc937f8845123a7e79aff55dc2960982a03f4cf9ba80743ea4fcb349c"} Jul 02 17:05:31.006136 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.006073 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:31.030073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.030031 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" podStartSLOduration=2.302348715 podStartE2EDuration="5.030015923s" totalImagesPullingTime="2.727667208s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:05:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:05:31.028073259 +0000 UTC m=+658.988422361" watchObservedRunningTime="2026-07-02 17:05:31.030015923 +0000 UTC m=+658.990364999" Jul 02 17:05:31.235930 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.235910 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:31.238605 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:05:31.238583 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" is forbidden: User \"system:node:ip-10-0-130-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-120.ec2.internal' and this object" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:31.310239 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.310176 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5gqvv\" (UniqueName: \"kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv\") pod \"f748cf95-fa8d-4919-afee-f79f23fdd134\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " Jul 02 17:05:31.310361 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.310268 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume\") pod \"f748cf95-fa8d-4919-afee-f79f23fdd134\" (UID: \"f748cf95-fa8d-4919-afee-f79f23fdd134\") " Jul 02 17:05:31.310520 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.310496 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "f748cf95-fa8d-4919-afee-f79f23fdd134" (UID: "f748cf95-fa8d-4919-afee-f79f23fdd134"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:05:31.312519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.312488 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv" (OuterVolumeSpecName: "kube-api-access-5gqvv") pod "f748cf95-fa8d-4919-afee-f79f23fdd134" (UID: "f748cf95-fa8d-4919-afee-f79f23fdd134"). InnerVolumeSpecName "kube-api-access-5gqvv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:05:31.410816 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.410785 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/f748cf95-fa8d-4919-afee-f79f23fdd134-extensions-socket-volume\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:05:31.410816 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:31.410809 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5gqvv\" (UniqueName: \"kubernetes.io/projected/f748cf95-fa8d-4919-afee-f79f23fdd134-kube-api-access-5gqvv\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:05:32.011453 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.011420 2565 generic.go:356] "Generic (PLEG): container finished" podID="f748cf95-fa8d-4919-afee-f79f23fdd134" containerID="e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0" exitCode=2 Jul 02 17:05:32.011881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.011465 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:32.011881 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.011527 2565 scope.go:117] "RemoveContainer" containerID="e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0" Jul 02 17:05:32.014228 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:05:32.014205 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" is forbidden: User \"system:node:ip-10-0-130-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-120.ec2.internal' and this object" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:32.020073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.020054 2565 scope.go:117] "RemoveContainer" containerID="e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0" Jul 02 17:05:32.020376 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:05:32.020359 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0\": container with ID starting with e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0 not found: ID does not exist" containerID="e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0" Jul 02 17:05:32.020434 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.020384 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0"} err="failed to get container status \"e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0\": rpc error: code = NotFound desc = could not find container \"e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0\": container with ID starting with e6eda391720ebf6d5be4729e415740736c3f9579f5d239d3e1a91307b57f69c0 not found: ID does not exist" Jul 02 17:05:32.020979 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:05:32.020955 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" is forbidden: User \"system:node:ip-10-0-130-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-120.ec2.internal' and this object" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:32.494077 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:05:32.494041 2565 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-sbsk2\" is forbidden: User \"system:node:ip-10-0-130-120.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-120.ec2.internal' and this object" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-sbsk2" Jul 02 17:05:32.494951 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:32.494932 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" path="/var/lib/kubelet/pods/f748cf95-fa8d-4919-afee-f79f23fdd134/volumes" Jul 02 17:05:42.014406 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:42.014372 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:53.940232 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:53.940199 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:53.940594 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:53.940428 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" podUID="28b54bdb-f582-4880-84ed-9b1c608b737e" containerName="manager" containerID="cri-o://9db9dd0bc937f8845123a7e79aff55dc2960982a03f4cf9ba80743ea4fcb349c" gracePeriod=10 Jul 02 17:05:54.111838 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.111709 2565 generic.go:356] "Generic (PLEG): container finished" podID="28b54bdb-f582-4880-84ed-9b1c608b737e" containerID="9db9dd0bc937f8845123a7e79aff55dc2960982a03f4cf9ba80743ea4fcb349c" exitCode=0 Jul 02 17:05:54.111838 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.111746 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" event={"ID":"28b54bdb-f582-4880-84ed-9b1c608b737e","Type":"ContainerDied","Data":"9db9dd0bc937f8845123a7e79aff55dc2960982a03f4cf9ba80743ea4fcb349c"} Jul 02 17:05:54.184673 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.184652 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:54.293513 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.293445 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume\") pod \"28b54bdb-f582-4880-84ed-9b1c608b737e\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " Jul 02 17:05:54.293513 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.293494 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6kn4l\" (UniqueName: \"kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l\") pod \"28b54bdb-f582-4880-84ed-9b1c608b737e\" (UID: \"28b54bdb-f582-4880-84ed-9b1c608b737e\") " Jul 02 17:05:54.293847 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.293821 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "28b54bdb-f582-4880-84ed-9b1c608b737e" (UID: "28b54bdb-f582-4880-84ed-9b1c608b737e"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:05:54.295507 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.295487 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l" (OuterVolumeSpecName: "kube-api-access-6kn4l") pod "28b54bdb-f582-4880-84ed-9b1c608b737e" (UID: "28b54bdb-f582-4880-84ed-9b1c608b737e"). InnerVolumeSpecName "kube-api-access-6kn4l". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:05:54.394133 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.394102 2565 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/28b54bdb-f582-4880-84ed-9b1c608b737e-extensions-socket-volume\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:05:54.394237 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:54.394145 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6kn4l\" (UniqueName: \"kubernetes.io/projected/28b54bdb-f582-4880-84ed-9b1c608b737e-kube-api-access-6kn4l\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:05:55.117541 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:55.117510 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" event={"ID":"28b54bdb-f582-4880-84ed-9b1c608b737e","Type":"ContainerDied","Data":"f6255ad7d85977b701636a5a85b6ed10a64de9c7bdb16c79eed3e7e7431ae863"} Jul 02 17:05:55.117885 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:55.117544 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8" Jul 02 17:05:55.117885 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:55.117570 2565 scope.go:117] "RemoveContainer" containerID="9db9dd0bc937f8845123a7e79aff55dc2960982a03f4cf9ba80743ea4fcb349c" Jul 02 17:05:55.136582 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:55.136559 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:55.140261 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:55.140243 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-44ws8"] Jul 02 17:05:56.498371 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:05:56.498337 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="28b54bdb-f582-4880-84ed-9b1c608b737e" path="/var/lib/kubelet/pods/28b54bdb-f582-4880-84ed-9b1c608b737e/volumes" Jul 02 17:06:10.456968 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.456935 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:10.457461 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457399 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" containerName="manager" Jul 02 17:06:10.457461 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457416 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" containerName="manager" Jul 02 17:06:10.457461 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457428 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="28b54bdb-f582-4880-84ed-9b1c608b737e" containerName="manager" Jul 02 17:06:10.457461 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457435 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="28b54bdb-f582-4880-84ed-9b1c608b737e" containerName="manager" Jul 02 17:06:10.457654 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457524 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f748cf95-fa8d-4919-afee-f79f23fdd134" containerName="manager" Jul 02 17:06:10.457654 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.457537 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="28b54bdb-f582-4880-84ed-9b1c608b737e" containerName="manager" Jul 02 17:06:10.460734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.460714 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.463184 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.463164 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-8c4pr\"" Jul 02 17:06:10.463184 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.463177 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 02 17:06:10.463357 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.463163 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 02 17:06:10.463357 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.463197 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 02 17:06:10.468413 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.468395 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:10.537223 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.537194 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.537401 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.537281 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdwsg\" (UniqueName: \"kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.559283 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.559254 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:10.638180 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.638150 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.638381 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.638217 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdwsg\" (UniqueName: \"kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.638813 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.638789 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.646343 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.646323 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdwsg\" (UniqueName: \"kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg\") pod \"limitador-limitador-64c94fd64f-z5bgb\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.771741 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.771662 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:10.903352 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.903282 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:10.905631 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:06:10.905601 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81237a28_7ba0_481e_837b_d75aa5752290.slice/crio-faf23b3cb3488d61d024ca9f029dbc82010ccf5911121d5c1317ca137c89af44 WatchSource:0}: Error finding container faf23b3cb3488d61d024ca9f029dbc82010ccf5911121d5c1317ca137c89af44: Status 404 returned error can't find the container with id faf23b3cb3488d61d024ca9f029dbc82010ccf5911121d5c1317ca137c89af44 Jul 02 17:06:10.907630 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:10.907610 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:06:11.185028 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:11.184986 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" event={"ID":"81237a28-7ba0-481e-837b-d75aa5752290","Type":"ContainerStarted","Data":"faf23b3cb3488d61d024ca9f029dbc82010ccf5911121d5c1317ca137c89af44"} Jul 02 17:06:14.201606 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:14.201571 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" event={"ID":"81237a28-7ba0-481e-837b-d75aa5752290","Type":"ContainerStarted","Data":"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925"} Jul 02 17:06:14.202022 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:14.201657 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:14.217927 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:14.217876 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" podStartSLOduration=1.496187003 podStartE2EDuration="4.21785553s" totalImagesPullingTime="2.721668527s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:06:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:06:14.217778778 +0000 UTC m=+702.178127851" watchObservedRunningTime="2026-07-02 17:06:14.21785553 +0000 UTC m=+702.178204607" Jul 02 17:06:25.097861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.097783 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:25.098276 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.097963 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" podUID="81237a28-7ba0-481e-837b-d75aa5752290" containerName="limitador" containerID="cri-o://615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925" gracePeriod=30 Jul 02 17:06:25.098715 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.098696 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:25.639137 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.639104 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:25.765377 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.765344 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file\") pod \"81237a28-7ba0-481e-837b-d75aa5752290\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " Jul 02 17:06:25.765554 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.765451 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdwsg\" (UniqueName: \"kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg\") pod \"81237a28-7ba0-481e-837b-d75aa5752290\" (UID: \"81237a28-7ba0-481e-837b-d75aa5752290\") " Jul 02 17:06:25.765806 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.765779 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file" (OuterVolumeSpecName: "config-file") pod "81237a28-7ba0-481e-837b-d75aa5752290" (UID: "81237a28-7ba0-481e-837b-d75aa5752290"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:06:25.767890 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.767857 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg" (OuterVolumeSpecName: "kube-api-access-fdwsg") pod "81237a28-7ba0-481e-837b-d75aa5752290" (UID: "81237a28-7ba0-481e-837b-d75aa5752290"). InnerVolumeSpecName "kube-api-access-fdwsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:06:25.866547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.866507 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdwsg\" (UniqueName: \"kubernetes.io/projected/81237a28-7ba0-481e-837b-d75aa5752290-kube-api-access-fdwsg\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:06:25.866547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:25.866546 2565 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/81237a28-7ba0-481e-837b-d75aa5752290-config-file\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:06:26.256585 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.256553 2565 generic.go:356] "Generic (PLEG): container finished" podID="81237a28-7ba0-481e-837b-d75aa5752290" containerID="615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925" exitCode=0 Jul 02 17:06:26.256585 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.256588 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" event={"ID":"81237a28-7ba0-481e-837b-d75aa5752290","Type":"ContainerDied","Data":"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925"} Jul 02 17:06:26.257065 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.256608 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" Jul 02 17:06:26.257065 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.256624 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-z5bgb" event={"ID":"81237a28-7ba0-481e-837b-d75aa5752290","Type":"ContainerDied","Data":"faf23b3cb3488d61d024ca9f029dbc82010ccf5911121d5c1317ca137c89af44"} Jul 02 17:06:26.257065 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.256644 2565 scope.go:117] "RemoveContainer" containerID="615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925" Jul 02 17:06:26.265376 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.265359 2565 scope.go:117] "RemoveContainer" containerID="615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925" Jul 02 17:06:26.265646 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:06:26.265623 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925\": container with ID starting with 615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925 not found: ID does not exist" containerID="615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925" Jul 02 17:06:26.265695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.265656 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925"} err="failed to get container status \"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925\": rpc error: code = NotFound desc = could not find container \"615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925\": container with ID starting with 615bd5dcf54e0adc9f03f070d6e5fe8638e7a32f818da04554296488bc4d2925 not found: ID does not exist" Jul 02 17:06:26.276404 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.276365 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:26.280605 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.280586 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-z5bgb"] Jul 02 17:06:26.494078 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:26.494034 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="81237a28-7ba0-481e-837b-d75aa5752290" path="/var/lib/kubelet/pods/81237a28-7ba0-481e-837b-d75aa5752290/volumes" Jul 02 17:06:28.370068 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.370039 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-69fb4594cf-zg854"] Jul 02 17:06:28.370554 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.370505 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81237a28-7ba0-481e-837b-d75aa5752290" containerName="limitador" Jul 02 17:06:28.370554 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.370522 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="81237a28-7ba0-481e-837b-d75aa5752290" containerName="limitador" Jul 02 17:06:28.370661 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.370607 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="81237a28-7ba0-481e-837b-d75aa5752290" containerName="limitador" Jul 02 17:06:28.374723 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.374701 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.377080 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.377050 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 17:06:28.377217 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.377205 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-dg72k\"" Jul 02 17:06:28.381380 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.381357 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-zg854"] Jul 02 17:06:28.490894 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.490864 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/364d72ea-37f3-4160-a398-40e96f751a0e-data\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.491134 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.490915 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ktl4q\" (UniqueName: \"kubernetes.io/projected/364d72ea-37f3-4160-a398-40e96f751a0e-kube-api-access-ktl4q\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.592233 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.592194 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ktl4q\" (UniqueName: \"kubernetes.io/projected/364d72ea-37f3-4160-a398-40e96f751a0e-kube-api-access-ktl4q\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.592428 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.592287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/364d72ea-37f3-4160-a398-40e96f751a0e-data\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.592627 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.592612 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/364d72ea-37f3-4160-a398-40e96f751a0e-data\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.600376 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.600351 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ktl4q\" (UniqueName: \"kubernetes.io/projected/364d72ea-37f3-4160-a398-40e96f751a0e-kube-api-access-ktl4q\") pod \"postgres-69fb4594cf-zg854\" (UID: \"364d72ea-37f3-4160-a398-40e96f751a0e\") " pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.686971 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.686935 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:28.820678 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:28.820598 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-69fb4594cf-zg854"] Jul 02 17:06:28.823228 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:06:28.823203 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod364d72ea_37f3_4160_a398_40e96f751a0e.slice/crio-5747799e54763ca7624b2393d573d67b642b701dc4e043bdf4e35f0b72b93173 WatchSource:0}: Error finding container 5747799e54763ca7624b2393d573d67b642b701dc4e043bdf4e35f0b72b93173: Status 404 returned error can't find the container with id 5747799e54763ca7624b2393d573d67b642b701dc4e043bdf4e35f0b72b93173 Jul 02 17:06:29.272279 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:29.272244 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-zg854" event={"ID":"364d72ea-37f3-4160-a398-40e96f751a0e","Type":"ContainerStarted","Data":"5747799e54763ca7624b2393d573d67b642b701dc4e043bdf4e35f0b72b93173"} Jul 02 17:06:34.346204 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:34.346182 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\"" Jul 02 17:06:35.303584 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:35.303539 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-69fb4594cf-zg854" event={"ID":"364d72ea-37f3-4160-a398-40e96f751a0e","Type":"ContainerStarted","Data":"b2b54bcd010b1246ed055a62e5b58f4eb313da214da1b767b07904b646aff2a7"} Jul 02 17:06:35.303756 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:35.303680 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:35.318559 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:35.318490 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/postgres-69fb4594cf-zg854" podStartSLOduration=1.799424702 podStartE2EDuration="7.318473395s" totalImagesPullingTime="5.519048693s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:06:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:06:35.317369911 +0000 UTC m=+723.277718986" watchObservedRunningTime="2026-07-02 17:06:35.318473395 +0000 UTC m=+723.278822515" Jul 02 17:06:41.338642 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:41.338611 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-69fb4594cf-zg854" Jul 02 17:06:48.196784 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.196749 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:06:48.387690 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.387663 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:06:48.387844 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.387755 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.390301 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.390283 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 17:06:48.390418 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.390318 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 17:06:48.391298 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.391281 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tsm99\"" Jul 02 17:06:48.395095 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.395008 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:06:48.421140 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.421094 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:06:48.421231 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.421190 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.566982 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.566924 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd786\" (UniqueName: \"kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.566982 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.566964 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf8ck\" (UniqueName: \"kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.567155 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.567010 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.567155 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.567038 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.668274 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.668255 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pf8ck\" (UniqueName: \"kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.668364 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.668292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.668364 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.668321 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.668433 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.668365 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rd786\" (UniqueName: \"kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.670882 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.670856 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.670963 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.670856 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.676897 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.676871 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf8ck\" (UniqueName: \"kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck\") pod \"maas-controller-7dbd5c7c9d-nprs5\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.677076 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.677057 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd786\" (UniqueName: \"kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786\") pod \"maas-controller-5d749c4bcb-w56vh\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.697940 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.697919 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:48.730089 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.730072 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:48.876432 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.876403 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:06:48.878956 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:06:48.878927 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod821c0b0b_229b_4b13_9ccd_1cb7fecd9054.slice/crio-946bf0b4d9d770df4038038603444cccd213c47fe28da1059b09a1092080b520 WatchSource:0}: Error finding container 946bf0b4d9d770df4038038603444cccd213c47fe28da1059b09a1092080b520: Status 404 returned error can't find the container with id 946bf0b4d9d770df4038038603444cccd213c47fe28da1059b09a1092080b520 Jul 02 17:06:48.897029 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:48.897009 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:06:48.899329 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:06:48.899304 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9e2ed045_bf44_490e_ae81_31622a12d814.slice/crio-9059933f2f660886f1f432278350416cdc6dffebec3112c35ffd4bebe72db5a4 WatchSource:0}: Error finding container 9059933f2f660886f1f432278350416cdc6dffebec3112c35ffd4bebe72db5a4: Status 404 returned error can't find the container with id 9059933f2f660886f1f432278350416cdc6dffebec3112c35ffd4bebe72db5a4 Jul 02 17:06:49.365087 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:49.365051 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" event={"ID":"821c0b0b-229b-4b13-9ccd-1cb7fecd9054","Type":"ContainerStarted","Data":"946bf0b4d9d770df4038038603444cccd213c47fe28da1059b09a1092080b520"} Jul 02 17:06:49.366164 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:49.366143 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" event={"ID":"9e2ed045-bf44-490e-ae81-31622a12d814","Type":"ContainerStarted","Data":"9059933f2f660886f1f432278350416cdc6dffebec3112c35ffd4bebe72db5a4"} Jul 02 17:06:52.381549 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.381511 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" event={"ID":"821c0b0b-229b-4b13-9ccd-1cb7fecd9054","Type":"ContainerStarted","Data":"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3"} Jul 02 17:06:52.381991 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.381658 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:06:52.383141 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.383098 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" event={"ID":"9e2ed045-bf44-490e-ae81-31622a12d814","Type":"ContainerStarted","Data":"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15"} Jul 02 17:06:52.383274 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.383161 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:06:52.398477 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.398441 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" podStartSLOduration=1.7804871100000002 podStartE2EDuration="4.398426278s" totalImagesPullingTime="2.617939168s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:06:52.397450445 +0000 UTC m=+740.357799530" watchObservedRunningTime="2026-07-02 17:06:52.398426278 +0000 UTC m=+740.358775344" Jul 02 17:06:52.419032 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:06:52.418936 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" podStartSLOduration=1.811165172 podStartE2EDuration="4.418917886s" totalImagesPullingTime="2.607752714s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:06:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:06:52.417325445 +0000 UTC m=+740.377674522" watchObservedRunningTime="2026-07-02 17:06:52.418917886 +0000 UTC m=+740.379266964" Jul 02 17:07:03.393220 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.393175 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:07:03.393712 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.393467 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:07:03.452160 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.452114 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:07:03.452347 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.452323 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" podUID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" containerName="manager" containerID="cri-o://b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3" gracePeriod=30 Jul 02 17:07:03.692803 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.692782 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:07:03.785715 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.785693 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert\") pod \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " Jul 02 17:07:03.785833 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.785765 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf8ck\" (UniqueName: \"kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck\") pod \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\" (UID: \"821c0b0b-229b-4b13-9ccd-1cb7fecd9054\") " Jul 02 17:07:03.787716 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.787694 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "821c0b0b-229b-4b13-9ccd-1cb7fecd9054" (UID: "821c0b0b-229b-4b13-9ccd-1cb7fecd9054"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:07:03.787866 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.787846 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck" (OuterVolumeSpecName: "kube-api-access-pf8ck") pod "821c0b0b-229b-4b13-9ccd-1cb7fecd9054" (UID: "821c0b0b-229b-4b13-9ccd-1cb7fecd9054"). InnerVolumeSpecName "kube-api-access-pf8ck". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:07:03.886520 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.886498 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-webhook-cert\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:07:03.886520 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:03.886517 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf8ck\" (UniqueName: \"kubernetes.io/projected/821c0b0b-229b-4b13-9ccd-1cb7fecd9054-kube-api-access-pf8ck\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:07:04.435929 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.435891 2565 generic.go:356] "Generic (PLEG): container finished" podID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" containerID="b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3" exitCode=0 Jul 02 17:07:04.436435 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.435955 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" Jul 02 17:07:04.436435 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.435990 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" event={"ID":"821c0b0b-229b-4b13-9ccd-1cb7fecd9054","Type":"ContainerDied","Data":"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3"} Jul 02 17:07:04.436435 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.436055 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7dbd5c7c9d-nprs5" event={"ID":"821c0b0b-229b-4b13-9ccd-1cb7fecd9054","Type":"ContainerDied","Data":"946bf0b4d9d770df4038038603444cccd213c47fe28da1059b09a1092080b520"} Jul 02 17:07:04.436435 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.436082 2565 scope.go:117] "RemoveContainer" containerID="b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3" Jul 02 17:07:04.444893 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.444872 2565 scope.go:117] "RemoveContainer" containerID="b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3" Jul 02 17:07:04.445193 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:07:04.445167 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3\": container with ID starting with b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3 not found: ID does not exist" containerID="b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3" Jul 02 17:07:04.445293 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.445204 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3"} err="failed to get container status \"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3\": rpc error: code = NotFound desc = could not find container \"b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3\": container with ID starting with b072e8fe278df21e5cf9aa9482fd120887b1435f5c4d2a8df0921b14c06d60f3 not found: ID does not exist" Jul 02 17:07:04.459303 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.459231 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:07:04.464571 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.464550 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7dbd5c7c9d-nprs5"] Jul 02 17:07:04.493449 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:04.493417 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" path="/var/lib/kubelet/pods/821c0b0b-229b-4b13-9ccd-1cb7fecd9054/volumes" Jul 02 17:07:23.702995 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.702962 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-647f5ff947-t7rql"] Jul 02 17:07:23.703578 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.703466 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" containerName="manager" Jul 02 17:07:23.703578 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.703486 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" containerName="manager" Jul 02 17:07:23.703578 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.703569 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="821c0b0b-229b-4b13-9ccd-1cb7fecd9054" containerName="manager" Jul 02 17:07:23.707738 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.707718 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.710731 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.710715 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-5k5c5\"" Jul 02 17:07:23.711570 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.711541 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jul 02 17:07:23.718935 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.718912 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-647f5ff947-t7rql"] Jul 02 17:07:23.840618 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.840589 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc000a1f-b533-484f-84cd-65679fa554e5-maas-api-tls\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.840750 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.840622 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z4cw6\" (UniqueName: \"kubernetes.io/projected/fc000a1f-b533-484f-84cd-65679fa554e5-kube-api-access-z4cw6\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.840750 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.840720 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc000a1f-b533-484f-84cd-65679fa554e5-openshift-service-ca\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.941418 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.941389 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc000a1f-b533-484f-84cd-65679fa554e5-openshift-service-ca\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.941567 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.941437 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc000a1f-b533-484f-84cd-65679fa554e5-maas-api-tls\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.941567 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.941480 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-z4cw6\" (UniqueName: \"kubernetes.io/projected/fc000a1f-b533-484f-84cd-65679fa554e5-kube-api-access-z4cw6\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.942050 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.942023 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/fc000a1f-b533-484f-84cd-65679fa554e5-openshift-service-ca\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.943931 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.943913 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/fc000a1f-b533-484f-84cd-65679fa554e5-maas-api-tls\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:23.950228 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:23.950181 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z4cw6\" (UniqueName: \"kubernetes.io/projected/fc000a1f-b533-484f-84cd-65679fa554e5-kube-api-access-z4cw6\") pod \"maas-api-647f5ff947-t7rql\" (UID: \"fc000a1f-b533-484f-84cd-65679fa554e5\") " pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:24.017452 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:24.017375 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:24.155895 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:24.155867 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-647f5ff947-t7rql"] Jul 02 17:07:24.158213 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:07:24.158178 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc000a1f_b533_484f_84cd_65679fa554e5.slice/crio-9d9620392e1c06b41781c0ca55a9ef6422c3216101d1904d8def9e6e9733d570 WatchSource:0}: Error finding container 9d9620392e1c06b41781c0ca55a9ef6422c3216101d1904d8def9e6e9733d570: Status 404 returned error can't find the container with id 9d9620392e1c06b41781c0ca55a9ef6422c3216101d1904d8def9e6e9733d570 Jul 02 17:07:24.526013 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:24.525974 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-647f5ff947-t7rql" event={"ID":"fc000a1f-b533-484f-84cd-65679fa554e5","Type":"ContainerStarted","Data":"9d9620392e1c06b41781c0ca55a9ef6422c3216101d1904d8def9e6e9733d570"} Jul 02 17:07:26.536808 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:26.536732 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-647f5ff947-t7rql" event={"ID":"fc000a1f-b533-484f-84cd-65679fa554e5","Type":"ContainerStarted","Data":"df2bccd548732be6fb9f3f2b711aa682e8ea7ca80821576eb4c892051725149e"} Jul 02 17:07:26.537258 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:26.536852 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:07:26.555165 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:26.555066 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-647f5ff947-t7rql" podStartSLOduration=1.514156642 podStartE2EDuration="3.555052579s" totalImagesPullingTime="2.040895937s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:07:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:07:26.553606182 +0000 UTC m=+774.513955259" watchObservedRunningTime="2026-07-02 17:07:26.555052579 +0000 UTC m=+774.515401655" Jul 02 17:07:32.545835 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:07:32.545811 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-647f5ff947-t7rql" Jul 02 17:08:04.172897 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.172819 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989"] Jul 02 17:08:04.176572 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.176556 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.179055 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.179030 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 02 17:08:04.180227 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.180196 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 02 17:08:04.180227 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.180214 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 02 17:08:04.180379 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.180263 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-gxrr7\"" Jul 02 17:08:04.187036 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.187015 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989"] Jul 02 17:08:04.258495 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258467 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd876\" (UniqueName: \"kubernetes.io/projected/db9e6493-f932-46b1-8dbd-041e9e712a58-kube-api-access-nd876\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.258680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258530 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.258680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258552 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.258680 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258655 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.258843 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258683 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.258843 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.258699 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9e6493-f932-46b1-8dbd-041e9e712a58-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359106 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359075 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359319 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359139 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359319 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359179 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359319 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359319 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359223 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9e6493-f932-46b1-8dbd-041e9e712a58-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359319 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359292 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nd876\" (UniqueName: \"kubernetes.io/projected/db9e6493-f932-46b1-8dbd-041e9e712a58-kube-api-access-nd876\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359560 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359495 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359560 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359519 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.359675 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.359657 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.361796 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.361773 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/db9e6493-f932-46b1-8dbd-041e9e712a58-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.362011 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.361994 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/db9e6493-f932-46b1-8dbd-041e9e712a58-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.367904 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.367881 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd876\" (UniqueName: \"kubernetes.io/projected/db9e6493-f932-46b1-8dbd-041e9e712a58-kube-api-access-nd876\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989\" (UID: \"db9e6493-f932-46b1-8dbd-041e9e712a58\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.487382 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.487315 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:04.618615 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.618592 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989"] Jul 02 17:08:04.620627 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:08:04.620589 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb9e6493_f932_46b1_8dbd_041e9e712a58.slice/crio-db0a172782f1a5f23aab9dcce93895b09e691490299cbeed2f494065627fade8 WatchSource:0}: Error finding container db0a172782f1a5f23aab9dcce93895b09e691490299cbeed2f494065627fade8: Status 404 returned error can't find the container with id db0a172782f1a5f23aab9dcce93895b09e691490299cbeed2f494065627fade8 Jul 02 17:08:04.701697 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:04.701667 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" event={"ID":"db9e6493-f932-46b1-8dbd-041e9e712a58","Type":"ContainerStarted","Data":"db0a172782f1a5f23aab9dcce93895b09e691490299cbeed2f494065627fade8"} Jul 02 17:08:06.711940 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:06.711898 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" event={"ID":"db9e6493-f932-46b1-8dbd-041e9e712a58","Type":"ContainerStarted","Data":"ae8c96cb7e5a14251989e84caadd2a5d7192eeaa8746abb2375c89ca5f8dd684"} Jul 02 17:08:06.734241 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:06.734192 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" podStartSLOduration=1.5401608690000002 podStartE2EDuration="2.734177464s" totalImagesPullingTime="1.194016595s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:08:04 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:08:06.732389023 +0000 UTC m=+814.692738099" watchObservedRunningTime="2026-07-02 17:08:06.734177464 +0000 UTC m=+814.694526560" Jul 02 17:08:10.959389 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:10.959335 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk"] Jul 02 17:08:10.962795 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:10.962770 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:10.965386 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:10.965367 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 02 17:08:10.974844 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:10.974818 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk"] Jul 02 17:08:11.022216 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022192 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.022345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022223 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qc9jn\" (UniqueName: \"kubernetes.io/projected/930e5897-8b16-4d08-93dd-66896d73f483-kube-api-access-qc9jn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.022345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022260 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.022345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022286 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930e5897-8b16-4d08-93dd-66896d73f483-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.022345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022326 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.022482 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.022343 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123031 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.122996 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123031 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123034 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123113 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123162 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qc9jn\" (UniqueName: \"kubernetes.io/projected/930e5897-8b16-4d08-93dd-66896d73f483-kube-api-access-qc9jn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123202 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123224 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930e5897-8b16-4d08-93dd-66896d73f483-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123462 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123446 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123533 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123514 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.123591 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.123553 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.125474 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.125454 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/930e5897-8b16-4d08-93dd-66896d73f483-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.125735 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.125718 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/930e5897-8b16-4d08-93dd-66896d73f483-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.130959 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.130938 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qc9jn\" (UniqueName: \"kubernetes.io/projected/930e5897-8b16-4d08-93dd-66896d73f483-kube-api-access-qc9jn\") pod \"e2e-distinct-simulated-kserve-59975d69f9-dh2pk\" (UID: \"930e5897-8b16-4d08-93dd-66896d73f483\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.274239 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.274147 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:11.406181 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.406150 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk"] Jul 02 17:08:11.409554 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:08:11.409528 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod930e5897_8b16_4d08_93dd_66896d73f483.slice/crio-84afd1f27534938d39ddf568f9d92d00591d1c541fc184f4392a8827052e6a31 WatchSource:0}: Error finding container 84afd1f27534938d39ddf568f9d92d00591d1c541fc184f4392a8827052e6a31: Status 404 returned error can't find the container with id 84afd1f27534938d39ddf568f9d92d00591d1c541fc184f4392a8827052e6a31 Jul 02 17:08:11.735508 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.735476 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" event={"ID":"930e5897-8b16-4d08-93dd-66896d73f483","Type":"ContainerStarted","Data":"600862c428d836a6059f87ac5a60e201958917ac830346e401f90e3473983061"} Jul 02 17:08:11.735685 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.735554 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" event={"ID":"930e5897-8b16-4d08-93dd-66896d73f483","Type":"ContainerStarted","Data":"84afd1f27534938d39ddf568f9d92d00591d1c541fc184f4392a8827052e6a31"} Jul 02 17:08:11.751482 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:11.751442 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" podStartSLOduration=1.554427879 podStartE2EDuration="1.75142671s" totalImagesPullingTime="196.998831ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:08:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:08:11.750490474 +0000 UTC m=+819.710839562" watchObservedRunningTime="2026-07-02 17:08:11.75142671 +0000 UTC m=+819.711775786" Jul 02 17:08:14.487853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:14.487822 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:14.488236 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:14.487871 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:14.500685 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:14.500662 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:14.759208 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:14.759111 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989" Jul 02 17:08:21.275017 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:21.274984 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:21.275411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:21.275043 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:21.287157 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:21.287137 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:08:21.787784 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:08:21.787759 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-dh2pk" Jul 02 17:09:32.513270 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:09:32.513193 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:09:32.516010 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:09:32.515991 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:10:59.900724 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:10:59.900638 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:10:59.901247 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:10:59.900882 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" podUID="9e2ed045-bf44-490e-ae81-31622a12d814" containerName="manager" containerID="cri-o://b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15" gracePeriod=30 Jul 02 17:11:00.143289 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.143266 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:11:00.259033 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.259005 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd786\" (UniqueName: \"kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786\") pod \"9e2ed045-bf44-490e-ae81-31622a12d814\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " Jul 02 17:11:00.259195 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.259040 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert\") pod \"9e2ed045-bf44-490e-ae81-31622a12d814\" (UID: \"9e2ed045-bf44-490e-ae81-31622a12d814\") " Jul 02 17:11:00.261141 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.261090 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "9e2ed045-bf44-490e-ae81-31622a12d814" (UID: "9e2ed045-bf44-490e-ae81-31622a12d814"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:11:00.261248 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.261224 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786" (OuterVolumeSpecName: "kube-api-access-rd786") pod "9e2ed045-bf44-490e-ae81-31622a12d814" (UID: "9e2ed045-bf44-490e-ae81-31622a12d814"). InnerVolumeSpecName "kube-api-access-rd786". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:11:00.360497 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.360470 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd786\" (UniqueName: \"kubernetes.io/projected/9e2ed045-bf44-490e-ae81-31622a12d814-kube-api-access-rd786\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:11:00.360497 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.360496 2565 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/9e2ed045-bf44-490e-ae81-31622a12d814-webhook-cert\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:11:00.483214 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.483179 2565 generic.go:356] "Generic (PLEG): container finished" podID="9e2ed045-bf44-490e-ae81-31622a12d814" containerID="b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15" exitCode=0 Jul 02 17:11:00.483320 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.483234 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" Jul 02 17:11:00.483320 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.483272 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" event={"ID":"9e2ed045-bf44-490e-ae81-31622a12d814","Type":"ContainerDied","Data":"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15"} Jul 02 17:11:00.483443 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.483344 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-w56vh" event={"ID":"9e2ed045-bf44-490e-ae81-31622a12d814","Type":"ContainerDied","Data":"9059933f2f660886f1f432278350416cdc6dffebec3112c35ffd4bebe72db5a4"} Jul 02 17:11:00.483443 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.483381 2565 scope.go:117] "RemoveContainer" containerID="b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15" Jul 02 17:11:00.492501 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.492483 2565 scope.go:117] "RemoveContainer" containerID="b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15" Jul 02 17:11:00.492770 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:11:00.492752 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15\": container with ID starting with b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15 not found: ID does not exist" containerID="b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15" Jul 02 17:11:00.492818 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.492779 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15"} err="failed to get container status \"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15\": rpc error: code = NotFound desc = could not find container \"b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15\": container with ID starting with b74c0b6bfdc15884a08a48cc8bee7da32bd43c894bad6b2c4d6311ad0197ac15 not found: ID does not exist" Jul 02 17:11:00.503652 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.503631 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:11:00.507135 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:00.507094 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-w56vh"] Jul 02 17:11:01.321572 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.321539 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-dff5l"] Jul 02 17:11:01.321911 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.321857 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9e2ed045-bf44-490e-ae81-31622a12d814" containerName="manager" Jul 02 17:11:01.321911 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.321868 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="9e2ed045-bf44-490e-ae81-31622a12d814" containerName="manager" Jul 02 17:11:01.321980 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.321918 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="9e2ed045-bf44-490e-ae81-31622a12d814" containerName="manager" Jul 02 17:11:01.325943 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.325921 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.328563 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.328542 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 02 17:11:01.328677 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.328540 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-tsm99\"" Jul 02 17:11:01.329634 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.329614 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 02 17:11:01.335815 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.335789 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-dff5l"] Jul 02 17:11:01.468707 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.468684 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6r7xg\" (UniqueName: \"kubernetes.io/projected/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-kube-api-access-6r7xg\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.468809 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.468737 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-webhook-cert\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.569823 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.569787 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-webhook-cert\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.569945 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.569920 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6r7xg\" (UniqueName: \"kubernetes.io/projected/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-kube-api-access-6r7xg\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.572345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.572293 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-webhook-cert\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.578448 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.578429 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6r7xg\" (UniqueName: \"kubernetes.io/projected/2314b0b8-93c4-4e0d-82b6-f7ca614c33ee-kube-api-access-6r7xg\") pod \"maas-controller-5d749c4bcb-dff5l\" (UID: \"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee\") " pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.637943 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.637924 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:01.765647 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:01.765622 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5d749c4bcb-dff5l"] Jul 02 17:11:01.767327 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:11:01.767301 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2314b0b8_93c4_4e0d_82b6_f7ca614c33ee.slice/crio-2d465b47e54814c73e5e46e8d67ec1305897ae9ab56e24474e514ba2c9c693b6 WatchSource:0}: Error finding container 2d465b47e54814c73e5e46e8d67ec1305897ae9ab56e24474e514ba2c9c693b6: Status 404 returned error can't find the container with id 2d465b47e54814c73e5e46e8d67ec1305897ae9ab56e24474e514ba2c9c693b6 Jul 02 17:11:02.494769 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:02.494672 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9e2ed045-bf44-490e-ae81-31622a12d814" path="/var/lib/kubelet/pods/9e2ed045-bf44-490e-ae81-31622a12d814/volumes" Jul 02 17:11:02.495245 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:02.495019 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" event={"ID":"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee","Type":"ContainerStarted","Data":"497e2212ce2dffc566b29dfc0487913f48ad751584afd549729fe59bb92d3c82"} Jul 02 17:11:02.495245 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:02.495069 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" event={"ID":"2314b0b8-93c4-4e0d-82b6-f7ca614c33ee","Type":"ContainerStarted","Data":"2d465b47e54814c73e5e46e8d67ec1305897ae9ab56e24474e514ba2c9c693b6"} Jul 02 17:11:02.495245 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:02.495164 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:11:02.511711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:02.511655 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" podStartSLOduration=1.040888807 podStartE2EDuration="1.51163766s" totalImagesPullingTime="470.748853ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:11:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:11:02.510304483 +0000 UTC m=+990.470653559" watchObservedRunningTime="2026-07-02 17:11:02.51163766 +0000 UTC m=+990.471986738" Jul 02 17:11:13.504796 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:11:13.504764 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5d749c4bcb-dff5l" Jul 02 17:14:32.540727 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:14:32.540694 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:14:32.545393 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:14:32.545363 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:15:00.150419 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.150369 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29716875-q84hw"] Jul 02 17:15:00.153803 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.153784 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:00.170458 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.170434 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29716875-q84hw"] Jul 02 17:15:00.245317 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.245283 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c7w5c\" (UniqueName: \"kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c\") pod \"maas-api-key-cleanup-29716875-q84hw\" (UID: \"c40c8da4-96f9-4b50-bc44-dda2762933a2\") " pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:00.345689 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.345658 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-c7w5c\" (UniqueName: \"kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c\") pod \"maas-api-key-cleanup-29716875-q84hw\" (UID: \"c40c8da4-96f9-4b50-bc44-dda2762933a2\") " pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:00.354816 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.354787 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c7w5c\" (UniqueName: \"kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c\") pod \"maas-api-key-cleanup-29716875-q84hw\" (UID: \"c40c8da4-96f9-4b50-bc44-dda2762933a2\") " pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:00.464076 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.464053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:00.592759 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.592661 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29716875-q84hw"] Jul 02 17:15:00.595705 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:15:00.595683 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc40c8da4_96f9_4b50_bc44_dda2762933a2.slice/crio-9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9 WatchSource:0}: Error finding container 9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9: Status 404 returned error can't find the container with id 9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9 Jul 02 17:15:00.597983 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:00.597965 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:15:01.555517 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:01.555404 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" event={"ID":"c40c8da4-96f9-4b50-bc44-dda2762933a2","Type":"ContainerStarted","Data":"9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9"} Jul 02 17:15:03.565555 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:03.565521 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" event={"ID":"c40c8da4-96f9-4b50-bc44-dda2762933a2","Type":"ContainerStarted","Data":"544eabe3ca2b8ed0272a8516cb0413f25e95d4cc71a590d7d02ac966ac631c1f"} Jul 02 17:15:03.582694 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:03.582560 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" podStartSLOduration=1.307963681 podStartE2EDuration="3.582543269s" totalImagesPullingTime="2.274579588s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:15:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:15:03.582258941 +0000 UTC m=+1231.542608016" watchObservedRunningTime="2026-07-02 17:15:03.582543269 +0000 UTC m=+1231.542892346" Jul 02 17:15:04.571141 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:04.571082 2565 generic.go:356] "Generic (PLEG): container finished" podID="c40c8da4-96f9-4b50-bc44-dda2762933a2" containerID="544eabe3ca2b8ed0272a8516cb0413f25e95d4cc71a590d7d02ac966ac631c1f" exitCode=0 Jul 02 17:15:04.571520 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:04.571155 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" event={"ID":"c40c8da4-96f9-4b50-bc44-dda2762933a2","Type":"ContainerDied","Data":"544eabe3ca2b8ed0272a8516cb0413f25e95d4cc71a590d7d02ac966ac631c1f"} Jul 02 17:15:05.706514 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:05.706492 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:15:05.889112 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:05.889057 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c7w5c\" (UniqueName: \"kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c\") pod \"c40c8da4-96f9-4b50-bc44-dda2762933a2\" (UID: \"c40c8da4-96f9-4b50-bc44-dda2762933a2\") " Jul 02 17:15:05.891189 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:05.891163 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c" (OuterVolumeSpecName: "kube-api-access-c7w5c") pod "c40c8da4-96f9-4b50-bc44-dda2762933a2" (UID: "c40c8da4-96f9-4b50-bc44-dda2762933a2"). InnerVolumeSpecName "kube-api-access-c7w5c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:15:05.989712 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:05.989689 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c7w5c\" (UniqueName: \"kubernetes.io/projected/c40c8da4-96f9-4b50-bc44-dda2762933a2-kube-api-access-c7w5c\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:15:06.585940 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:06.585900 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" event={"ID":"c40c8da4-96f9-4b50-bc44-dda2762933a2","Type":"ContainerDied","Data":"9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9"} Jul 02 17:15:06.586069 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:06.585986 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9208adf88af042fe748ca6fc0256ada9f896e98651f1199f686f28b1539cb5f9" Jul 02 17:15:06.586069 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:15:06.585917 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716875-q84hw" Jul 02 17:19:32.568108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:19:32.568077 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:19:32.572627 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:19:32.572604 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:22:54.797086 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.797037 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:22:54.797896 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.797872 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c40c8da4-96f9-4b50-bc44-dda2762933a2" containerName="cleanup" Jul 02 17:22:54.797987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.797900 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="c40c8da4-96f9-4b50-bc44-dda2762933a2" containerName="cleanup" Jul 02 17:22:54.798447 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.798053 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="c40c8da4-96f9-4b50-bc44-dda2762933a2" containerName="cleanup" Jul 02 17:22:54.809438 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.809400 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.811734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.811705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:22:54.812021 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.811999 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-c42a90b6-serving-cert\"" Jul 02 17:22:54.866669 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.866641 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4g89v\" (UniqueName: \"kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.866809 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.866679 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.866809 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.866718 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.968186 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.968155 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.968345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.968259 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4g89v\" (UniqueName: \"kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.968345 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.968287 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.968868 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.968846 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.970721 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.970700 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:54.977162 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:54.977105 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4g89v\" (UniqueName: \"kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v\") pod \"maas-api-e2e-ait-c42a90b6-8469776659-vg7pg\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:55.122608 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:55.122528 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:55.259146 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:55.259098 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:22:55.261743 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:22:55.261709 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94d932fc_facc_4833_a422_33ee6bb64f02.slice/crio-dc537844aaf64e7d88730226d8586b72133f8bd027cbf0e676ffdee39d3e849b WatchSource:0}: Error finding container dc537844aaf64e7d88730226d8586b72133f8bd027cbf0e676ffdee39d3e849b: Status 404 returned error can't find the container with id dc537844aaf64e7d88730226d8586b72133f8bd027cbf0e676ffdee39d3e849b Jul 02 17:22:55.263049 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:55.263032 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:22:55.694734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:55.694698 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" event={"ID":"94d932fc-facc-4833-a422-33ee6bb64f02","Type":"ContainerStarted","Data":"dc537844aaf64e7d88730226d8586b72133f8bd027cbf0e676ffdee39d3e849b"} Jul 02 17:22:56.700457 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:56.700396 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" event={"ID":"94d932fc-facc-4833-a422-33ee6bb64f02","Type":"ContainerStarted","Data":"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae"} Jul 02 17:22:56.700862 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:56.700492 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:22:56.720485 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:22:56.720438 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" podStartSLOduration=2.244539982 podStartE2EDuration="2.720423843s" totalImagesPullingTime="475.883861ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:22:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:22:56.718955023 +0000 UTC m=+1704.679304112" watchObservedRunningTime="2026-07-02 17:22:56.720423843 +0000 UTC m=+1704.680772918" Jul 02 17:23:00.272586 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.272555 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:23:00.273065 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.272770 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" podUID="94d932fc-facc-4833-a422-33ee6bb64f02" containerName="maas-api" containerID="cri-o://4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae" gracePeriod=30 Jul 02 17:23:00.278846 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.278822 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:23:00.522314 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.522291 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:23:00.623004 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.622931 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca\") pod \"94d932fc-facc-4833-a422-33ee6bb64f02\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " Jul 02 17:23:00.623004 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.622970 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls\") pod \"94d932fc-facc-4833-a422-33ee6bb64f02\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " Jul 02 17:23:00.623274 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.623028 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4g89v\" (UniqueName: \"kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v\") pod \"94d932fc-facc-4833-a422-33ee6bb64f02\" (UID: \"94d932fc-facc-4833-a422-33ee6bb64f02\") " Jul 02 17:23:00.623365 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.623340 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "94d932fc-facc-4833-a422-33ee6bb64f02" (UID: "94d932fc-facc-4833-a422-33ee6bb64f02"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:00.625290 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.625251 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v" (OuterVolumeSpecName: "kube-api-access-4g89v") pod "94d932fc-facc-4833-a422-33ee6bb64f02" (UID: "94d932fc-facc-4833-a422-33ee6bb64f02"). InnerVolumeSpecName "kube-api-access-4g89v". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:00.625410 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.625327 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "94d932fc-facc-4833-a422-33ee6bb64f02" (UID: "94d932fc-facc-4833-a422-33ee6bb64f02"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:23:00.719829 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.719796 2565 generic.go:356] "Generic (PLEG): container finished" podID="94d932fc-facc-4833-a422-33ee6bb64f02" containerID="4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae" exitCode=0 Jul 02 17:23:00.719988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.719839 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" Jul 02 17:23:00.719988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.719855 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" event={"ID":"94d932fc-facc-4833-a422-33ee6bb64f02","Type":"ContainerDied","Data":"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae"} Jul 02 17:23:00.719988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.719903 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg" event={"ID":"94d932fc-facc-4833-a422-33ee6bb64f02","Type":"ContainerDied","Data":"dc537844aaf64e7d88730226d8586b72133f8bd027cbf0e676ffdee39d3e849b"} Jul 02 17:23:00.719988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.719926 2565 scope.go:117] "RemoveContainer" containerID="4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae" Jul 02 17:23:00.727410 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.727381 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/94d932fc-facc-4833-a422-33ee6bb64f02-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:00.727410 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.727413 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/94d932fc-facc-4833-a422-33ee6bb64f02-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:00.727667 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.727430 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4g89v\" (UniqueName: \"kubernetes.io/projected/94d932fc-facc-4833-a422-33ee6bb64f02-kube-api-access-4g89v\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:00.735579 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.735562 2565 scope.go:117] "RemoveContainer" containerID="4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae" Jul 02 17:23:00.735835 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:23:00.735816 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae\": container with ID starting with 4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae not found: ID does not exist" containerID="4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae" Jul 02 17:23:00.735902 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.735843 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae"} err="failed to get container status \"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae\": rpc error: code = NotFound desc = could not find container \"4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae\": container with ID starting with 4a9635f7ab562b61b19019920b8beabdad5d33c8d3c9ae843770fe8663d133ae not found: ID does not exist" Jul 02 17:23:00.747568 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.747545 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:23:00.751947 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:00.751926 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-c42a90b6-8469776659-vg7pg"] Jul 02 17:23:02.494258 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:02.494226 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="94d932fc-facc-4833-a422-33ee6bb64f02" path="/var/lib/kubelet/pods/94d932fc-facc-4833-a422-33ee6bb64f02/volumes" Jul 02 17:23:07.498840 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.498809 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:07.499364 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.499346 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="94d932fc-facc-4833-a422-33ee6bb64f02" containerName="maas-api" Jul 02 17:23:07.499443 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.499368 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="94d932fc-facc-4833-a422-33ee6bb64f02" containerName="maas-api" Jul 02 17:23:07.499494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.499482 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="94d932fc-facc-4833-a422-33ee6bb64f02" containerName="maas-api" Jul 02 17:23:07.502601 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.502579 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.505305 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.505284 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-ait-e36c7a3e-serving-cert\"" Jul 02 17:23:07.515400 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.515364 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:07.595173 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.595113 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.595534 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.595499 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djln7\" (UniqueName: \"kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.595692 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.595668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.696453 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.696405 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.696453 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.696454 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djln7\" (UniqueName: \"kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.696721 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.696559 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.697179 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.697153 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.699094 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.699069 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.705670 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.705647 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djln7\" (UniqueName: \"kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7\") pod \"maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.816383 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.816295 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:07.968960 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:07.968919 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:07.971261 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:23:07.971228 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb76bfa9b_7f26_4106_9710_5d0b9be4ec25.slice/crio-0bc92db3139f49845786c4ca09789e6a149985a95f5000d653387ddb70365646 WatchSource:0}: Error finding container 0bc92db3139f49845786c4ca09789e6a149985a95f5000d653387ddb70365646: Status 404 returned error can't find the container with id 0bc92db3139f49845786c4ca09789e6a149985a95f5000d653387ddb70365646 Jul 02 17:23:08.756668 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:08.756636 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" event={"ID":"b76bfa9b-7f26-4106-9710-5d0b9be4ec25","Type":"ContainerStarted","Data":"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59"} Jul 02 17:23:08.757073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:08.756692 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" event={"ID":"b76bfa9b-7f26-4106-9710-5d0b9be4ec25","Type":"ContainerStarted","Data":"0bc92db3139f49845786c4ca09789e6a149985a95f5000d653387ddb70365646"} Jul 02 17:23:08.757073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:08.756728 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:08.775764 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:08.775715 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" podStartSLOduration=1.344275924 podStartE2EDuration="1.77569965s" totalImagesPullingTime="431.423726ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:23:07 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:23:08.774888582 +0000 UTC m=+1716.735237673" watchObservedRunningTime="2026-07-02 17:23:08.77569965 +0000 UTC m=+1716.736048726" Jul 02 17:23:13.009555 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.009527 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:13.009941 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.009760 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" podUID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" containerName="maas-api" containerID="cri-o://e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59" gracePeriod=30 Jul 02 17:23:13.015626 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.015607 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:13.257391 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.257361 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:13.351503 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.351419 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls\") pod \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " Jul 02 17:23:13.351503 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.351474 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca\") pod \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " Jul 02 17:23:13.351684 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.351530 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djln7\" (UniqueName: \"kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7\") pod \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\" (UID: \"b76bfa9b-7f26-4106-9710-5d0b9be4ec25\") " Jul 02 17:23:13.351882 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.351851 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b76bfa9b-7f26-4106-9710-5d0b9be4ec25" (UID: "b76bfa9b-7f26-4106-9710-5d0b9be4ec25"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:13.353753 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.353725 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b76bfa9b-7f26-4106-9710-5d0b9be4ec25" (UID: "b76bfa9b-7f26-4106-9710-5d0b9be4ec25"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:23:13.353868 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.353844 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7" (OuterVolumeSpecName: "kube-api-access-djln7") pod "b76bfa9b-7f26-4106-9710-5d0b9be4ec25" (UID: "b76bfa9b-7f26-4106-9710-5d0b9be4ec25"). InnerVolumeSpecName "kube-api-access-djln7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:13.452895 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.452868 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:13.452895 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.452899 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:13.453062 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.452910 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djln7\" (UniqueName: \"kubernetes.io/projected/b76bfa9b-7f26-4106-9710-5d0b9be4ec25-kube-api-access-djln7\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:13.780519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.780483 2565 generic.go:356] "Generic (PLEG): container finished" podID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" containerID="e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59" exitCode=0 Jul 02 17:23:13.780723 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.780529 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" Jul 02 17:23:13.780723 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.780552 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" event={"ID":"b76bfa9b-7f26-4106-9710-5d0b9be4ec25","Type":"ContainerDied","Data":"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59"} Jul 02 17:23:13.780723 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.780602 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw" event={"ID":"b76bfa9b-7f26-4106-9710-5d0b9be4ec25","Type":"ContainerDied","Data":"0bc92db3139f49845786c4ca09789e6a149985a95f5000d653387ddb70365646"} Jul 02 17:23:13.780723 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.780629 2565 scope.go:117] "RemoveContainer" containerID="e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59" Jul 02 17:23:13.789638 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.789619 2565 scope.go:117] "RemoveContainer" containerID="e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59" Jul 02 17:23:13.789886 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:23:13.789868 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59\": container with ID starting with e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59 not found: ID does not exist" containerID="e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59" Jul 02 17:23:13.789937 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.789896 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59"} err="failed to get container status \"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59\": rpc error: code = NotFound desc = could not find container \"e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59\": container with ID starting with e969281601c2ce1f6307b973b778dbaf5ca31172092c57cd7fef9c6474f1aa59 not found: ID does not exist" Jul 02 17:23:13.801380 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.801352 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:13.805793 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:13.805772 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-ait-e36c7a3e-9c69c89cf-982xw"] Jul 02 17:23:14.495333 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:14.495300 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" path="/var/lib/kubelet/pods/b76bfa9b-7f26-4106-9710-5d0b9be4ec25/volumes" Jul 02 17:23:26.407583 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.407488 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:26.408108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.408058 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" containerName="maas-api" Jul 02 17:23:26.408108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.408080 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" containerName="maas-api" Jul 02 17:23:26.408267 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.408189 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b76bfa9b-7f26-4106-9710-5d0b9be4ec25" containerName="maas-api" Jul 02 17:23:26.412205 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.412184 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.414952 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.414932 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-derive-a52f81ac-serving-cert\"" Jul 02 17:23:26.425538 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.425515 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:26.464167 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.464140 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.464339 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.464211 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lxxjf\" (UniqueName: \"kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.464339 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.464316 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.565357 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.565323 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lxxjf\" (UniqueName: \"kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.565543 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.565395 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.565543 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.565438 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.565636 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:23:26.565543 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-derive-a52f81ac-serving-cert: secret "maas-api-e2e-derive-a52f81ac-serving-cert" not found Jul 02 17:23:26.565636 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:23:26.565595 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls podName:3bd6b939-855f-4cf1-b4b7-7a2307002e83 nodeName:}" failed. No retries permitted until 2026-07-02 17:23:27.065579146 +0000 UTC m=+1735.025928200 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls") pod "maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" (UID: "3bd6b939-855f-4cf1-b4b7-7a2307002e83") : secret "maas-api-e2e-derive-a52f81ac-serving-cert" not found Jul 02 17:23:26.566181 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.566162 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:26.575085 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:26.575065 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lxxjf\" (UniqueName: \"kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:27.069185 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:27.069151 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:27.071675 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:27.071652 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") pod \"maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:27.323418 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:27.323321 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:27.461272 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:27.461155 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:27.464185 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:23:27.464154 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3bd6b939_855f_4cf1_b4b7_7a2307002e83.slice/crio-3b96aecff0f996c25355b2de3cba07060ab57b07bfe93f46809a9247c40ca61d WatchSource:0}: Error finding container 3b96aecff0f996c25355b2de3cba07060ab57b07bfe93f46809a9247c40ca61d: Status 404 returned error can't find the container with id 3b96aecff0f996c25355b2de3cba07060ab57b07bfe93f46809a9247c40ca61d Jul 02 17:23:27.846575 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:27.846548 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" event={"ID":"3bd6b939-855f-4cf1-b4b7-7a2307002e83","Type":"ContainerStarted","Data":"3b96aecff0f996c25355b2de3cba07060ab57b07bfe93f46809a9247c40ca61d"} Jul 02 17:23:28.852299 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:28.852268 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" event={"ID":"3bd6b939-855f-4cf1-b4b7-7a2307002e83","Type":"ContainerStarted","Data":"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff"} Jul 02 17:23:28.852722 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:28.852343 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:28.871419 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:28.871375 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" podStartSLOduration=2.512651327 podStartE2EDuration="2.871357524s" totalImagesPullingTime="358.706197ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:23:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:23:28.8701629 +0000 UTC m=+1736.830511993" watchObservedRunningTime="2026-07-02 17:23:28.871357524 +0000 UTC m=+1736.831706599" Jul 02 17:23:31.180480 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.180447 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:31.180884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.180673 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" podUID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" containerName="maas-api" containerID="cri-o://8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff" gracePeriod=30 Jul 02 17:23:31.430539 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.430471 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:31.504563 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.504535 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca\") pod \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " Jul 02 17:23:31.504738 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.504627 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") pod \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " Jul 02 17:23:31.504738 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.504650 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lxxjf\" (UniqueName: \"kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf\") pod \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\" (UID: \"3bd6b939-855f-4cf1-b4b7-7a2307002e83\") " Jul 02 17:23:31.504979 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.504951 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "3bd6b939-855f-4cf1-b4b7-7a2307002e83" (UID: "3bd6b939-855f-4cf1-b4b7-7a2307002e83"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:23:31.506862 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.506841 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "3bd6b939-855f-4cf1-b4b7-7a2307002e83" (UID: "3bd6b939-855f-4cf1-b4b7-7a2307002e83"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:23:31.506946 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.506861 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf" (OuterVolumeSpecName: "kube-api-access-lxxjf") pod "3bd6b939-855f-4cf1-b4b7-7a2307002e83" (UID: "3bd6b939-855f-4cf1-b4b7-7a2307002e83"). InnerVolumeSpecName "kube-api-access-lxxjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:23:31.605487 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.605450 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/3bd6b939-855f-4cf1-b4b7-7a2307002e83-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:31.605487 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.605481 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lxxjf\" (UniqueName: \"kubernetes.io/projected/3bd6b939-855f-4cf1-b4b7-7a2307002e83-kube-api-access-lxxjf\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:31.605487 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.605496 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/3bd6b939-855f-4cf1-b4b7-7a2307002e83-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:23:31.867307 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.867207 2565 generic.go:356] "Generic (PLEG): container finished" podID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" containerID="8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff" exitCode=0 Jul 02 17:23:31.867307 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.867244 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" event={"ID":"3bd6b939-855f-4cf1-b4b7-7a2307002e83","Type":"ContainerDied","Data":"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff"} Jul 02 17:23:31.867307 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.867264 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" Jul 02 17:23:31.867307 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.867284 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46" event={"ID":"3bd6b939-855f-4cf1-b4b7-7a2307002e83","Type":"ContainerDied","Data":"3b96aecff0f996c25355b2de3cba07060ab57b07bfe93f46809a9247c40ca61d"} Jul 02 17:23:31.867307 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.867306 2565 scope.go:117] "RemoveContainer" containerID="8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff" Jul 02 17:23:31.875941 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.875923 2565 scope.go:117] "RemoveContainer" containerID="8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff" Jul 02 17:23:31.876221 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:23:31.876202 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff\": container with ID starting with 8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff not found: ID does not exist" containerID="8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff" Jul 02 17:23:31.876305 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.876232 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff"} err="failed to get container status \"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff\": rpc error: code = NotFound desc = could not find container \"8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff\": container with ID starting with 8f1741c1c265bfe841f026fd671dac92fca2efc0855f8f31600e389cb674e5ff not found: ID does not exist" Jul 02 17:23:31.887932 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.887910 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:31.894348 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:31.894326 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-derive-a52f81ac-84ddfc65cd-xhg46"] Jul 02 17:23:32.495524 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:23:32.495493 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" path="/var/lib/kubelet/pods/3bd6b939-855f-4cf1-b4b7-7a2307002e83/volumes" Jul 02 17:24:32.602428 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:24:32.602402 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:24:32.607001 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:24:32.606977 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:25:15.135482 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.135448 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:15.136074 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.135958 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" containerName="maas-api" Jul 02 17:25:15.136074 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.135974 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" containerName="maas-api" Jul 02 17:25:15.136074 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.136054 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="3bd6b939-855f-4cf1-b4b7-7a2307002e83" containerName="maas-api" Jul 02 17:25:15.139509 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.139488 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.141900 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.141878 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-7d46b667-serving-cert\"" Jul 02 17:25:15.149428 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.149401 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:15.231541 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.231507 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.231830 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.231800 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.231971 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.231937 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fdshm\" (UniqueName: \"kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.332699 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.332670 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.332879 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.332717 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.332879 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.332766 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fdshm\" (UniqueName: \"kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.333492 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.333464 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.335541 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.335513 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.341048 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.341025 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fdshm\" (UniqueName: \"kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm\") pod \"maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.451933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.451909 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:15.618695 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:15.618665 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:15.620642 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:25:15.620617 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6fb49044_f1f0_4cf5_b3fd_ddfb553391e3.slice/crio-8a0fbee0337709cf09f4777839c55fe2a99aff3bccb5b7a07f00bb8dce11cc63 WatchSource:0}: Error finding container 8a0fbee0337709cf09f4777839c55fe2a99aff3bccb5b7a07f00bb8dce11cc63: Status 404 returned error can't find the container with id 8a0fbee0337709cf09f4777839c55fe2a99aff3bccb5b7a07f00bb8dce11cc63 Jul 02 17:25:16.324500 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:16.324410 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" event={"ID":"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3","Type":"ContainerStarted","Data":"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13"} Jul 02 17:25:16.324500 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:16.324468 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" event={"ID":"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3","Type":"ContainerStarted","Data":"8a0fbee0337709cf09f4777839c55fe2a99aff3bccb5b7a07f00bb8dce11cc63"} Jul 02 17:25:16.324907 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:16.324547 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:16.341770 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:16.341701 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" podStartSLOduration=0.938172934 podStartE2EDuration="1.341679642s" totalImagesPullingTime="403.506708ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:25:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:25:16.33878216 +0000 UTC m=+1844.299131237" watchObservedRunningTime="2026-07-02 17:25:16.341679642 +0000 UTC m=+1844.302028771" Jul 02 17:25:22.334144 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:22.334096 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:27.283400 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.283363 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:27.283822 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.283586 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerName="maas-api" containerID="cri-o://6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13" gracePeriod=30 Jul 02 17:25:27.330230 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.330194 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.36:8443/health\": dial tcp 10.134.0.36:8443: connect: connection refused" Jul 02 17:25:27.525688 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.525659 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:27.639638 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.639556 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fdshm\" (UniqueName: \"kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm\") pod \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " Jul 02 17:25:27.639638 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.639632 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca\") pod \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " Jul 02 17:25:27.639858 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.639671 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls\") pod \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\" (UID: \"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3\") " Jul 02 17:25:27.640563 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.640535 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" (UID: "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:25:27.641964 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.641933 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" (UID: "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:25:27.642064 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.641980 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm" (OuterVolumeSpecName: "kube-api-access-fdshm") pod "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" (UID: "6fb49044-f1f0-4cf5-b3fd-ddfb553391e3"). InnerVolumeSpecName "kube-api-access-fdshm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:25:27.740634 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.740599 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fdshm\" (UniqueName: \"kubernetes.io/projected/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-kube-api-access-fdshm\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:25:27.740634 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.740629 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:25:27.740634 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:27.740642 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:25:28.377548 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.377511 2565 generic.go:356] "Generic (PLEG): container finished" podID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerID="6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13" exitCode=0 Jul 02 17:25:28.377987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.377553 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" Jul 02 17:25:28.377987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.377601 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" event={"ID":"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3","Type":"ContainerDied","Data":"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13"} Jul 02 17:25:28.377987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.377650 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8" event={"ID":"6fb49044-f1f0-4cf5-b3fd-ddfb553391e3","Type":"ContainerDied","Data":"8a0fbee0337709cf09f4777839c55fe2a99aff3bccb5b7a07f00bb8dce11cc63"} Jul 02 17:25:28.377987 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.377672 2565 scope.go:117] "RemoveContainer" containerID="6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13" Jul 02 17:25:28.387091 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.387078 2565 scope.go:117] "RemoveContainer" containerID="6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13" Jul 02 17:25:28.387366 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:25:28.387350 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13\": container with ID starting with 6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13 not found: ID does not exist" containerID="6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13" Jul 02 17:25:28.387417 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.387374 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13"} err="failed to get container status \"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13\": rpc error: code = NotFound desc = could not find container \"6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13\": container with ID starting with 6383a663eb5c03b58f9d1adbae639aab763dce77160d894697c84d949b1cec13 not found: ID does not exist" Jul 02 17:25:28.398439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.398419 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:28.404248 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.404219 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-7d46b667-6876f4766f-7z4g8"] Jul 02 17:25:28.494204 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:28.494173 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" path="/var/lib/kubelet/pods/6fb49044-f1f0-4cf5-b3fd-ddfb553391e3/volumes" Jul 02 17:25:55.185859 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.185781 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:25:55.186358 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.186111 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerName="maas-api" Jul 02 17:25:55.186358 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.186155 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerName="maas-api" Jul 02 17:25:55.186358 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.186231 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="6fb49044-f1f0-4cf5-b3fd-ddfb553391e3" containerName="maas-api" Jul 02 17:25:55.190422 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.190405 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.193263 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.193240 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-mt-3245e5b9-serving-cert\"" Jul 02 17:25:55.198391 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.198369 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:25:55.288008 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.287971 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.288192 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.288016 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.288192 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.288150 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tzmzf\" (UniqueName: \"kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.389096 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.389052 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tzmzf\" (UniqueName: \"kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.389300 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.389160 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.389300 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.389200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.389377 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:25:55.389340 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-mt-3245e5b9-serving-cert: secret "maas-api-e2e-mt-3245e5b9-serving-cert" not found Jul 02 17:25:55.389419 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:25:55.389412 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls podName:a6480539-2a41-47b9-86cd-4b3ea421cb38 nodeName:}" failed. No retries permitted until 2026-07-02 17:25:55.889384442 +0000 UTC m=+1883.849733498 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls") pod "maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" (UID: "a6480539-2a41-47b9-86cd-4b3ea421cb38") : secret "maas-api-e2e-mt-3245e5b9-serving-cert" not found Jul 02 17:25:55.389824 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.389803 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.398415 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.398388 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tzmzf\" (UniqueName: \"kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.893234 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.893200 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:55.895742 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:55.895710 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") pod \"maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:56.102072 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:56.102038 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:56.244598 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:56.244572 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:25:56.246632 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:25:56.246593 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6480539_2a41_47b9_86cd_4b3ea421cb38.slice/crio-b5f910f8b859d648fecaeaa51969a66eb7b67e5d63c299874394b6b1daba49da WatchSource:0}: Error finding container b5f910f8b859d648fecaeaa51969a66eb7b67e5d63c299874394b6b1daba49da: Status 404 returned error can't find the container with id b5f910f8b859d648fecaeaa51969a66eb7b67e5d63c299874394b6b1daba49da Jul 02 17:25:56.517405 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:56.517366 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" event={"ID":"a6480539-2a41-47b9-86cd-4b3ea421cb38","Type":"ContainerStarted","Data":"b5f910f8b859d648fecaeaa51969a66eb7b67e5d63c299874394b6b1daba49da"} Jul 02 17:25:57.523439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:57.523401 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" event={"ID":"a6480539-2a41-47b9-86cd-4b3ea421cb38","Type":"ContainerStarted","Data":"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba"} Jul 02 17:25:57.523851 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:57.523553 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:25:57.540458 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:25:57.540410 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" podStartSLOduration=1.919967199 podStartE2EDuration="2.540395496s" totalImagesPullingTime="620.428297ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:25:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:25:57.539346773 +0000 UTC m=+1885.499695850" watchObservedRunningTime="2026-07-02 17:25:57.540395496 +0000 UTC m=+1885.500744572" Jul 02 17:26:03.533385 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:03.533355 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:26:27.693818 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.693786 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:26:27.694390 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.693982 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" podUID="a6480539-2a41-47b9-86cd-4b3ea421cb38" containerName="maas-api" containerID="cri-o://b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba" gracePeriod=30 Jul 02 17:26:27.949778 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.949711 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:26:27.990556 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.990520 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca\") pod \"a6480539-2a41-47b9-86cd-4b3ea421cb38\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " Jul 02 17:26:27.990721 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.990606 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") pod \"a6480539-2a41-47b9-86cd-4b3ea421cb38\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " Jul 02 17:26:27.990721 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.990632 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tzmzf\" (UniqueName: \"kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf\") pod \"a6480539-2a41-47b9-86cd-4b3ea421cb38\" (UID: \"a6480539-2a41-47b9-86cd-4b3ea421cb38\") " Jul 02 17:26:27.990940 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.990918 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a6480539-2a41-47b9-86cd-4b3ea421cb38" (UID: "a6480539-2a41-47b9-86cd-4b3ea421cb38"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:26:27.992903 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.992870 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf" (OuterVolumeSpecName: "kube-api-access-tzmzf") pod "a6480539-2a41-47b9-86cd-4b3ea421cb38" (UID: "a6480539-2a41-47b9-86cd-4b3ea421cb38"). InnerVolumeSpecName "kube-api-access-tzmzf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:26:27.992903 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:27.992870 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a6480539-2a41-47b9-86cd-4b3ea421cb38" (UID: "a6480539-2a41-47b9-86cd-4b3ea421cb38"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:26:28.091513 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.091480 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a6480539-2a41-47b9-86cd-4b3ea421cb38-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:26:28.091513 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.091510 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a6480539-2a41-47b9-86cd-4b3ea421cb38-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:26:28.091703 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.091520 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tzmzf\" (UniqueName: \"kubernetes.io/projected/a6480539-2a41-47b9-86cd-4b3ea421cb38-kube-api-access-tzmzf\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:26:28.660060 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.660027 2565 generic.go:356] "Generic (PLEG): container finished" podID="a6480539-2a41-47b9-86cd-4b3ea421cb38" containerID="b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba" exitCode=0 Jul 02 17:26:28.660249 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.660071 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" Jul 02 17:26:28.660249 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.660141 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" event={"ID":"a6480539-2a41-47b9-86cd-4b3ea421cb38","Type":"ContainerDied","Data":"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba"} Jul 02 17:26:28.660249 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.660192 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr" event={"ID":"a6480539-2a41-47b9-86cd-4b3ea421cb38","Type":"ContainerDied","Data":"b5f910f8b859d648fecaeaa51969a66eb7b67e5d63c299874394b6b1daba49da"} Jul 02 17:26:28.660249 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.660215 2565 scope.go:117] "RemoveContainer" containerID="b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba" Jul 02 17:26:28.668609 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.668590 2565 scope.go:117] "RemoveContainer" containerID="b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba" Jul 02 17:26:28.668893 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:26:28.668862 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba\": container with ID starting with b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba not found: ID does not exist" containerID="b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba" Jul 02 17:26:28.668933 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.668900 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba"} err="failed to get container status \"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba\": rpc error: code = NotFound desc = could not find container \"b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba\": container with ID starting with b07f4832a66c35c2bafb2f6bae0d09fc4b910f6456cc84f388b4fb0394123eba not found: ID does not exist" Jul 02 17:26:28.680376 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.680351 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:26:28.684663 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:28.684643 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-mt-3245e5b9-586b66bc96-6t4hr"] Jul 02 17:26:30.494530 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:26:30.494497 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a6480539-2a41-47b9-86cd-4b3ea421cb38" path="/var/lib/kubelet/pods/a6480539-2a41-47b9-86cd-4b3ea421cb38/volumes" Jul 02 17:28:18.963111 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.963065 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:28:18.965852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.963710 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a6480539-2a41-47b9-86cd-4b3ea421cb38" containerName="maas-api" Jul 02 17:28:18.965852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.963731 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a6480539-2a41-47b9-86cd-4b3ea421cb38" containerName="maas-api" Jul 02 17:28:18.965852 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.963830 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a6480539-2a41-47b9-86cd-4b3ea421cb38" containerName="maas-api" Jul 02 17:28:18.966988 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.966968 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:18.969471 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.969445 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-a-bb1d81-serving-cert\"" Jul 02 17:28:18.978107 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:18.978076 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:28:19.075402 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.075363 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.075592 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.075447 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vbsn6\" (UniqueName: \"kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.075592 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.075474 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.176021 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.175974 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vbsn6\" (UniqueName: \"kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.176021 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.176020 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.176295 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.176054 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.176674 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.176647 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.178638 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.178612 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.184072 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.184052 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vbsn6\" (UniqueName: \"kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6\") pod \"maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.278533 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.278446 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:19.420281 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.420253 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:28:19.423619 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:28:19.423580 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb5d43d17_26c8_462b_bf50_24564d1e6503.slice/crio-32106780fb4edc2c348b7023c40faef90a3c57bbc86f628d596b12be1a086168 WatchSource:0}: Error finding container 32106780fb4edc2c348b7023c40faef90a3c57bbc86f628d596b12be1a086168: Status 404 returned error can't find the container with id 32106780fb4edc2c348b7023c40faef90a3c57bbc86f628d596b12be1a086168 Jul 02 17:28:19.424992 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:19.424973 2565 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 02 17:28:20.173400 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:20.173352 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" event={"ID":"b5d43d17-26c8-462b-bf50-24564d1e6503","Type":"ContainerStarted","Data":"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e"} Jul 02 17:28:20.173910 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:20.173458 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" event={"ID":"b5d43d17-26c8-462b-bf50-24564d1e6503","Type":"ContainerStarted","Data":"32106780fb4edc2c348b7023c40faef90a3c57bbc86f628d596b12be1a086168"} Jul 02 17:28:20.173910 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:20.173525 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:20.190832 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:20.190781 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" podStartSLOduration=1.6722592779999998 podStartE2EDuration="2.190764072s" totalImagesPullingTime="518.504794ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:28:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:28:20.189235504 +0000 UTC m=+2028.149584613" watchObservedRunningTime="2026-07-02 17:28:20.190764072 +0000 UTC m=+2028.151113148" Jul 02 17:28:26.183416 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:26.183376 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:28:27.075001 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.074966 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:28:27.078682 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.078666 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.081031 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.080997 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-inf-b-50b097-serving-cert\"" Jul 02 17:28:27.087591 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.087568 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:28:27.251936 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.251894 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dhn6w\" (UniqueName: \"kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.252398 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.251952 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.252398 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.252137 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.353199 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.353078 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dhn6w\" (UniqueName: \"kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.353199 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.353172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.353419 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.353233 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.353823 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.353805 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.355830 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.355799 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.361422 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.361393 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dhn6w\" (UniqueName: \"kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w\") pod \"maas-api-e2e-inf-b-50b097-75778b97b9-q5f67\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.390576 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.390529 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:27.530888 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:27.530863 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:28:27.533458 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:28:27.533426 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2f30e1a_c29a_4393_a9e6_d2ca5a9f9caa.slice/crio-af52755fea74fa95fb8afe1fd90823e232481bfe0facfe3d43b0ea57c48bbe10 WatchSource:0}: Error finding container af52755fea74fa95fb8afe1fd90823e232481bfe0facfe3d43b0ea57c48bbe10: Status 404 returned error can't find the container with id af52755fea74fa95fb8afe1fd90823e232481bfe0facfe3d43b0ea57c48bbe10 Jul 02 17:28:28.210706 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:28.210668 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" event={"ID":"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa","Type":"ContainerStarted","Data":"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0"} Jul 02 17:28:28.210900 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:28.210724 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" event={"ID":"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa","Type":"ContainerStarted","Data":"af52755fea74fa95fb8afe1fd90823e232481bfe0facfe3d43b0ea57c48bbe10"} Jul 02 17:28:28.210900 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:28.210806 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:28:28.228352 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:28.228302 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" podStartSLOduration=0.896409488 podStartE2EDuration="1.228286636s" totalImagesPullingTime="331.877148ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:28:27 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:28:28.226403519 +0000 UTC m=+2036.186752607" watchObservedRunningTime="2026-07-02 17:28:28.228286636 +0000 UTC m=+2036.188635760" Jul 02 17:28:34.220785 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:28:34.220757 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:29:32.630174 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:29:32.630149 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:29:32.636545 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:29:32.636525 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:30:00.136729 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.136693 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9"] Jul 02 17:30:00.140948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.140931 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-29716890-xdnvc"] Jul 02 17:30:00.141070 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.141053 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:00.144415 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.144393 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx"] Jul 02 17:30:00.144635 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.144545 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:00.148311 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.148281 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9"] Jul 02 17:30:00.148439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.148384 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:00.150057 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.150034 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx"] Jul 02 17:30:00.159766 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.159735 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29716890-xdnvc"] Jul 02 17:30:00.188743 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.188713 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4mmsz\" (UniqueName: \"kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz\") pod \"maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9\" (UID: \"d449e448-9f35-4a91-a4d3-57e7a7906330\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:00.188743 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.188744 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5pcg\" (UniqueName: \"kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg\") pod \"maas-api-key-cleanup-29716890-xdnvc\" (UID: \"b1ad6447-7275-45e6-8267-c67e01e36d60\") " pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:00.188914 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.188777 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ghvd7\" (UniqueName: \"kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7\") pod \"maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx\" (UID: \"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:00.289458 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.289429 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4mmsz\" (UniqueName: \"kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz\") pod \"maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9\" (UID: \"d449e448-9f35-4a91-a4d3-57e7a7906330\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:00.289458 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.289463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t5pcg\" (UniqueName: \"kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg\") pod \"maas-api-key-cleanup-29716890-xdnvc\" (UID: \"b1ad6447-7275-45e6-8267-c67e01e36d60\") " pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:00.289637 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.289608 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ghvd7\" (UniqueName: \"kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7\") pod \"maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx\" (UID: \"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:00.298411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.298385 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4mmsz\" (UniqueName: \"kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz\") pod \"maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9\" (UID: \"d449e448-9f35-4a91-a4d3-57e7a7906330\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:00.298411 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.298406 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ghvd7\" (UniqueName: \"kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7\") pod \"maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx\" (UID: \"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb\") " pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:00.298599 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.298482 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5pcg\" (UniqueName: \"kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg\") pod \"maas-api-key-cleanup-29716890-xdnvc\" (UID: \"b1ad6447-7275-45e6-8267-c67e01e36d60\") " pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:00.456992 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.456964 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:00.467280 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.467260 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:00.473445 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.473420 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:00.817471 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.817448 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9"] Jul 02 17:30:00.819293 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:30:00.819263 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd449e448_9f35_4a91_a4d3_57e7a7906330.slice/crio-f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e WatchSource:0}: Error finding container f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e: Status 404 returned error can't find the container with id f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e Jul 02 17:30:00.847570 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.847545 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx"] Jul 02 17:30:00.850428 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:00.850408 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-key-cleanup-29716890-xdnvc"] Jul 02 17:30:00.854384 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:30:00.854358 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd8a0f2e0_3f6c_431a_96cf_f171e3167bdb.slice/crio-a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589 WatchSource:0}: Error finding container a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589: Status 404 returned error can't find the container with id a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589 Jul 02 17:30:00.854581 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:30:00.854563 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb1ad6447_7275_45e6_8267_c67e01e36d60.slice/crio-161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a WatchSource:0}: Error finding container 161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a: Status 404 returned error can't find the container with id 161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a Jul 02 17:30:01.654814 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.654776 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" event={"ID":"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb","Type":"ContainerStarted","Data":"ed9eec3a21a71915066b80074fcc55cd4184e5f37571930d2f19f117ada116ba"} Jul 02 17:30:01.655252 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.654841 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" event={"ID":"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb","Type":"ContainerStarted","Data":"a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589"} Jul 02 17:30:01.656401 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.656371 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" event={"ID":"b1ad6447-7275-45e6-8267-c67e01e36d60","Type":"ContainerStarted","Data":"97ab8dc4dfe6cf83e778dce1b5ded3158766c9fc4972edcbb40d5f757af27220"} Jul 02 17:30:01.656698 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.656421 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" event={"ID":"b1ad6447-7275-45e6-8267-c67e01e36d60","Type":"ContainerStarted","Data":"161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a"} Jul 02 17:30:01.657965 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.657940 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" event={"ID":"d449e448-9f35-4a91-a4d3-57e7a7906330","Type":"ContainerStarted","Data":"f09a9b14eaa8b511c03686c5f2a8410e837ff3b1984098d95e4864ca2c0a15c2"} Jul 02 17:30:01.658165 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.657980 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" event={"ID":"d449e448-9f35-4a91-a4d3-57e7a7906330","Type":"ContainerStarted","Data":"f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e"} Jul 02 17:30:01.669947 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.669892 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" podStartSLOduration=1.6698774159999998 podStartE2EDuration="1.669877416s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:30:01.66862901 +0000 UTC m=+2129.628978097" watchObservedRunningTime="2026-07-02 17:30:01.669877416 +0000 UTC m=+2129.630226470" Jul 02 17:30:01.683358 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.683319 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" podStartSLOduration=1.683304798 podStartE2EDuration="1.683304798s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:30:01.681872773 +0000 UTC m=+2129.642221851" watchObservedRunningTime="2026-07-02 17:30:01.683304798 +0000 UTC m=+2129.643653873" Jul 02 17:30:01.695525 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:01.695492 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" podStartSLOduration=1.69547877 podStartE2EDuration="1.69547877s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:30:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:30:01.6950785 +0000 UTC m=+2129.655427577" watchObservedRunningTime="2026-07-02 17:30:01.69547877 +0000 UTC m=+2129.655827845" Jul 02 17:30:02.664313 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.664277 2565 generic.go:356] "Generic (PLEG): container finished" podID="d449e448-9f35-4a91-a4d3-57e7a7906330" containerID="f09a9b14eaa8b511c03686c5f2a8410e837ff3b1984098d95e4864ca2c0a15c2" exitCode=0 Jul 02 17:30:02.664757 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.664321 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" event={"ID":"d449e448-9f35-4a91-a4d3-57e7a7906330","Type":"ContainerDied","Data":"f09a9b14eaa8b511c03686c5f2a8410e837ff3b1984098d95e4864ca2c0a15c2"} Jul 02 17:30:02.666159 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.666106 2565 generic.go:356] "Generic (PLEG): container finished" podID="d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" containerID="ed9eec3a21a71915066b80074fcc55cd4184e5f37571930d2f19f117ada116ba" exitCode=0 Jul 02 17:30:02.666303 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.666170 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" event={"ID":"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb","Type":"ContainerDied","Data":"ed9eec3a21a71915066b80074fcc55cd4184e5f37571930d2f19f117ada116ba"} Jul 02 17:30:02.668054 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.668034 2565 generic.go:356] "Generic (PLEG): container finished" podID="b1ad6447-7275-45e6-8267-c67e01e36d60" containerID="97ab8dc4dfe6cf83e778dce1b5ded3158766c9fc4972edcbb40d5f757af27220" exitCode=0 Jul 02 17:30:02.668187 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:02.668060 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" event={"ID":"b1ad6447-7275-45e6-8267-c67e01e36d60","Type":"ContainerDied","Data":"97ab8dc4dfe6cf83e778dce1b5ded3158766c9fc4972edcbb40d5f757af27220"} Jul 02 17:30:03.832211 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.832180 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:03.860792 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.860769 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:03.862861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.862846 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:03.921486 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.921453 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4mmsz\" (UniqueName: \"kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz\") pod \"d449e448-9f35-4a91-a4d3-57e7a7906330\" (UID: \"d449e448-9f35-4a91-a4d3-57e7a7906330\") " Jul 02 17:30:03.921657 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.921523 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t5pcg\" (UniqueName: \"kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg\") pod \"b1ad6447-7275-45e6-8267-c67e01e36d60\" (UID: \"b1ad6447-7275-45e6-8267-c67e01e36d60\") " Jul 02 17:30:03.921657 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.921626 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ghvd7\" (UniqueName: \"kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7\") pod \"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb\" (UID: \"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb\") " Jul 02 17:30:03.923837 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.923772 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg" (OuterVolumeSpecName: "kube-api-access-t5pcg") pod "b1ad6447-7275-45e6-8267-c67e01e36d60" (UID: "b1ad6447-7275-45e6-8267-c67e01e36d60"). InnerVolumeSpecName "kube-api-access-t5pcg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:30:03.923837 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.923813 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7" (OuterVolumeSpecName: "kube-api-access-ghvd7") pod "d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" (UID: "d8a0f2e0-3f6c-431a-96cf-f171e3167bdb"). InnerVolumeSpecName "kube-api-access-ghvd7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:30:03.923837 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:03.923827 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz" (OuterVolumeSpecName: "kube-api-access-4mmsz") pod "d449e448-9f35-4a91-a4d3-57e7a7906330" (UID: "d449e448-9f35-4a91-a4d3-57e7a7906330"). InnerVolumeSpecName "kube-api-access-4mmsz". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:30:04.022702 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.022667 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t5pcg\" (UniqueName: \"kubernetes.io/projected/b1ad6447-7275-45e6-8267-c67e01e36d60-kube-api-access-t5pcg\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:04.022702 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.022693 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ghvd7\" (UniqueName: \"kubernetes.io/projected/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb-kube-api-access-ghvd7\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:04.022702 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.022704 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4mmsz\" (UniqueName: \"kubernetes.io/projected/d449e448-9f35-4a91-a4d3-57e7a7906330-kube-api-access-4mmsz\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:04.678829 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.678803 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" Jul 02 17:30:04.679019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.678842 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9" event={"ID":"d449e448-9f35-4a91-a4d3-57e7a7906330","Type":"ContainerDied","Data":"f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e"} Jul 02 17:30:04.679019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.678897 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f7209f08cacff0ab8d50969586c0f28c015bc62c50a4c062b81b263b836ce61e" Jul 02 17:30:04.680247 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.680225 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" Jul 02 17:30:04.680395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.680228 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx" event={"ID":"d8a0f2e0-3f6c-431a-96cf-f171e3167bdb","Type":"ContainerDied","Data":"a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589"} Jul 02 17:30:04.680395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.680332 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a26783f76e946790b5a269e23d2758010dcc55aa37812e170dd2b8f1457d8589" Jul 02 17:30:04.681594 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.681569 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" event={"ID":"b1ad6447-7275-45e6-8267-c67e01e36d60","Type":"ContainerDied","Data":"161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a"} Jul 02 17:30:04.681728 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.681603 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-key-cleanup-29716890-xdnvc" Jul 02 17:30:04.681728 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:04.681611 2565 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="161ee417d7dcbc3a4892a9b9993296e5972a465e219b692da6af16be1a84b77a" Jul 02 17:30:13.014800 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.014766 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:30:13.015245 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.015015 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" podUID="b5d43d17-26c8-462b-bf50-24564d1e6503" containerName="maas-api" containerID="cri-o://a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e" gracePeriod=30 Jul 02 17:30:13.039313 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.039287 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx"] Jul 02 17:30:13.042285 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.042255 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-a-bb1d81-29716890-dq4bx"] Jul 02 17:30:13.457033 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.457007 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:30:13.505059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.505034 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca\") pod \"b5d43d17-26c8-462b-bf50-24564d1e6503\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " Jul 02 17:30:13.505257 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.505101 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls\") pod \"b5d43d17-26c8-462b-bf50-24564d1e6503\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " Jul 02 17:30:13.505257 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.505155 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vbsn6\" (UniqueName: \"kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6\") pod \"b5d43d17-26c8-462b-bf50-24564d1e6503\" (UID: \"b5d43d17-26c8-462b-bf50-24564d1e6503\") " Jul 02 17:30:13.505456 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.505433 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "b5d43d17-26c8-462b-bf50-24564d1e6503" (UID: "b5d43d17-26c8-462b-bf50-24564d1e6503"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:30:13.507334 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.507313 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6" (OuterVolumeSpecName: "kube-api-access-vbsn6") pod "b5d43d17-26c8-462b-bf50-24564d1e6503" (UID: "b5d43d17-26c8-462b-bf50-24564d1e6503"). InnerVolumeSpecName "kube-api-access-vbsn6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:30:13.507486 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.507466 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "b5d43d17-26c8-462b-bf50-24564d1e6503" (UID: "b5d43d17-26c8-462b-bf50-24564d1e6503"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:30:13.606861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.606564 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/b5d43d17-26c8-462b-bf50-24564d1e6503-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:13.606861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.606604 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/b5d43d17-26c8-462b-bf50-24564d1e6503-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:13.606861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.606620 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vbsn6\" (UniqueName: \"kubernetes.io/projected/b5d43d17-26c8-462b-bf50-24564d1e6503-kube-api-access-vbsn6\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:13.731374 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.731335 2565 generic.go:356] "Generic (PLEG): container finished" podID="b5d43d17-26c8-462b-bf50-24564d1e6503" containerID="a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e" exitCode=0 Jul 02 17:30:13.731537 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.731392 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" Jul 02 17:30:13.731537 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.731425 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" event={"ID":"b5d43d17-26c8-462b-bf50-24564d1e6503","Type":"ContainerDied","Data":"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e"} Jul 02 17:30:13.731537 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.731486 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw" event={"ID":"b5d43d17-26c8-462b-bf50-24564d1e6503","Type":"ContainerDied","Data":"32106780fb4edc2c348b7023c40faef90a3c57bbc86f628d596b12be1a086168"} Jul 02 17:30:13.731537 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.731506 2565 scope.go:117] "RemoveContainer" containerID="a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e" Jul 02 17:30:13.740541 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.740523 2565 scope.go:117] "RemoveContainer" containerID="a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e" Jul 02 17:30:13.740789 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:30:13.740771 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e\": container with ID starting with a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e not found: ID does not exist" containerID="a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e" Jul 02 17:30:13.740834 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.740803 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e"} err="failed to get container status \"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e\": rpc error: code = NotFound desc = could not find container \"a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e\": container with ID starting with a1e02032948b85fb3c383fc2f0e0030c5f7218b4129fe6b674b29d5e30d7417e not found: ID does not exist" Jul 02 17:30:13.751057 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.751033 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:30:13.758226 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:13.758201 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-a-bb1d81-78d9795899-pxdpw"] Jul 02 17:30:14.493948 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:14.493917 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b5d43d17-26c8-462b-bf50-24564d1e6503" path="/var/lib/kubelet/pods/b5d43d17-26c8-462b-bf50-24564d1e6503/volumes" Jul 02 17:30:14.494349 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:14.494260 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" path="/var/lib/kubelet/pods/d8a0f2e0-3f6c-431a-96cf-f171e3167bdb/volumes" Jul 02 17:30:55.897139 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:55.897091 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:30:55.897743 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:55.897404 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" podUID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" containerName="maas-api" containerID="cri-o://97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0" gracePeriod=30 Jul 02 17:30:55.931467 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:55.931440 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9"] Jul 02 17:30:55.933252 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:55.933228 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-key-cleanup-e2e-inf-b-50b097-29716890-pdhm9"] Jul 02 17:30:56.152112 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.152056 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:30:56.200744 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.200716 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dhn6w\" (UniqueName: \"kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w\") pod \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " Jul 02 17:30:56.200939 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.200755 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca\") pod \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " Jul 02 17:30:56.200939 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.200786 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls\") pod \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\" (UID: \"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa\") " Jul 02 17:30:56.201172 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.201147 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" (UID: "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:30:56.202883 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.202858 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" (UID: "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:30:56.202985 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.202938 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w" (OuterVolumeSpecName: "kube-api-access-dhn6w") pod "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" (UID: "f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa"). InnerVolumeSpecName "kube-api-access-dhn6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:30:56.301734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.301704 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dhn6w\" (UniqueName: \"kubernetes.io/projected/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-kube-api-access-dhn6w\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:56.301734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.301731 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:56.301734 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.301741 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:30:56.494079 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.494049 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d449e448-9f35-4a91-a4d3-57e7a7906330" path="/var/lib/kubelet/pods/d449e448-9f35-4a91-a4d3-57e7a7906330/volumes" Jul 02 17:30:56.929037 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.929001 2565 generic.go:356] "Generic (PLEG): container finished" podID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" containerID="97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0" exitCode=0 Jul 02 17:30:56.929494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.929045 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" Jul 02 17:30:56.929494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.929096 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" event={"ID":"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa","Type":"ContainerDied","Data":"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0"} Jul 02 17:30:56.929494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.929169 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67" event={"ID":"f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa","Type":"ContainerDied","Data":"af52755fea74fa95fb8afe1fd90823e232481bfe0facfe3d43b0ea57c48bbe10"} Jul 02 17:30:56.929494 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.929191 2565 scope.go:117] "RemoveContainer" containerID="97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0" Jul 02 17:30:56.937595 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.937576 2565 scope.go:117] "RemoveContainer" containerID="97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0" Jul 02 17:30:56.937880 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:30:56.937861 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0\": container with ID starting with 97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0 not found: ID does not exist" containerID="97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0" Jul 02 17:30:56.937929 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.937889 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0"} err="failed to get container status \"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0\": rpc error: code = NotFound desc = could not find container \"97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0\": container with ID starting with 97ce0268b1480a36ab5cb013e6d99611ef4c1224948ecd78a8cc0096fc28c1c0 not found: ID does not exist" Jul 02 17:30:56.945320 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.945292 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:30:56.947294 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:56.947272 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-inf-b-50b097-75778b97b9-q5f67"] Jul 02 17:30:58.493737 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:30:58.493704 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" path="/var/lib/kubelet/pods/f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa/volumes" Jul 02 17:31:06.054222 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054178 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054560 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" containerName="maas-api" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054572 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" containerName="maas-api" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054587 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b5d43d17-26c8-462b-bf50-24564d1e6503" containerName="maas-api" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054592 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b5d43d17-26c8-462b-bf50-24564d1e6503" containerName="maas-api" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054599 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b1ad6447-7275-45e6-8267-c67e01e36d60" containerName="cleanup" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054605 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="b1ad6447-7275-45e6-8267-c67e01e36d60" containerName="cleanup" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054614 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d449e448-9f35-4a91-a4d3-57e7a7906330" containerName="cleanup" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054621 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d449e448-9f35-4a91-a4d3-57e7a7906330" containerName="cleanup" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054639 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" containerName="cleanup" Jul 02 17:31:06.054711 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054647 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" containerName="cleanup" Jul 02 17:31:06.055019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054725 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b5d43d17-26c8-462b-bf50-24564d1e6503" containerName="maas-api" Jul 02 17:31:06.055019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054733 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="b1ad6447-7275-45e6-8267-c67e01e36d60" containerName="cleanup" Jul 02 17:31:06.055019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054740 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="f2f30e1a-c29a-4393-a9e6-d2ca5a9f9caa" containerName="maas-api" Jul 02 17:31:06.055019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054745 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d8a0f2e0-3f6c-431a-96cf-f171e3167bdb" containerName="cleanup" Jul 02 17:31:06.055019 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.054750 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="d449e448-9f35-4a91-a4d3-57e7a7906330" containerName="cleanup" Jul 02 17:31:06.059102 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.059083 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.061475 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.061456 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-a-28df41-serving-cert\"" Jul 02 17:31:06.069294 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.069273 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:06.195555 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.195517 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.195555 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.195551 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.195800 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.195585 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vw6w\" (UniqueName: \"kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.296853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.296823 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.296853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.296856 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.297183 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.296895 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9vw6w\" (UniqueName: \"kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.297183 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:06.297011 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-api-a-28df41-serving-cert: secret "maas-api-e2e-api-a-28df41-serving-cert" not found Jul 02 17:31:06.297183 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:06.297098 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls podName:e4018913-c621-48df-b46b-67a0d8f14fc6 nodeName:}" failed. No retries permitted until 2026-07-02 17:31:06.797077635 +0000 UTC m=+2194.757426704 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls") pod "maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" (UID: "e4018913-c621-48df-b46b-67a0d8f14fc6") : secret "maas-api-e2e-api-a-28df41-serving-cert" not found Jul 02 17:31:06.297626 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.297608 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.310838 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.310770 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vw6w\" (UniqueName: \"kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.801180 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.801111 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.803641 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.803616 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") pod \"maas-api-e2e-api-a-28df41-6c479485f5-rzkk6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:06.970659 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:06.970620 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:07.114388 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:07.114352 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:07.119963 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:31:07.119863 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode4018913_c621_48df_b46b_67a0d8f14fc6.slice/crio-510b2c37428e02a182a3be7abccbfbd676e8cb5889fd2e7233b230a477bd5458 WatchSource:0}: Error finding container 510b2c37428e02a182a3be7abccbfbd676e8cb5889fd2e7233b230a477bd5458: Status 404 returned error can't find the container with id 510b2c37428e02a182a3be7abccbfbd676e8cb5889fd2e7233b230a477bd5458 Jul 02 17:31:07.983328 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:07.983291 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" event={"ID":"e4018913-c621-48df-b46b-67a0d8f14fc6","Type":"ContainerStarted","Data":"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674"} Jul 02 17:31:07.983531 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:07.983351 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" event={"ID":"e4018913-c621-48df-b46b-67a0d8f14fc6","Type":"ContainerStarted","Data":"510b2c37428e02a182a3be7abccbfbd676e8cb5889fd2e7233b230a477bd5458"} Jul 02 17:31:07.983531 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:07.983375 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:08.000402 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:08.000347 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" podStartSLOduration=1.499548358 podStartE2EDuration="2.000332307s" totalImagesPullingTime="500.783949ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:07.99820317 +0000 UTC m=+2195.958552252" watchObservedRunningTime="2026-07-02 17:31:08.000332307 +0000 UTC m=+2195.960681395" Jul 02 17:31:13.995136 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:13.995097 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:15.377193 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.377161 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:15.380884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.380859 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.383197 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.383171 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-api-b-a295d5-serving-cert\"" Jul 02 17:31:15.390510 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.390484 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:15.484747 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.484716 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.484934 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.484782 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mrck7\" (UniqueName: \"kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.484934 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.484838 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.585923 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.585886 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mrck7\" (UniqueName: \"kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.586108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.585936 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.586108 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.585974 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.586678 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.586613 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.588638 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.588612 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.594716 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.594695 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mrck7\" (UniqueName: \"kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7\") pod \"maas-api-e2e-api-b-a295d5-fb4497f77-4978p\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.693822 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.693788 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:15.832050 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:15.832019 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:15.834397 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:31:15.834371 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod62b84d32_f2a0_4a5d_898b_05f9e96cb39e.slice/crio-75bb4b989f11812e5a618e92e5b2689c73f41b3dbc3d18485afe55bebc886543 WatchSource:0}: Error finding container 75bb4b989f11812e5a618e92e5b2689c73f41b3dbc3d18485afe55bebc886543: Status 404 returned error can't find the container with id 75bb4b989f11812e5a618e92e5b2689c73f41b3dbc3d18485afe55bebc886543 Jul 02 17:31:16.023364 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:16.023278 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" event={"ID":"62b84d32-f2a0-4a5d-898b-05f9e96cb39e","Type":"ContainerStarted","Data":"75bb4b989f11812e5a618e92e5b2689c73f41b3dbc3d18485afe55bebc886543"} Jul 02 17:31:17.029311 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:17.029275 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" event={"ID":"62b84d32-f2a0-4a5d-898b-05f9e96cb39e","Type":"ContainerStarted","Data":"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce"} Jul 02 17:31:17.029741 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:17.029382 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:17.046740 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:17.046692 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" podStartSLOduration=1.7377487249999999 podStartE2EDuration="2.046675349s" totalImagesPullingTime="308.926624ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:17.045941983 +0000 UTC m=+2205.006291059" watchObservedRunningTime="2026-07-02 17:31:17.046675349 +0000 UTC m=+2205.007024428" Jul 02 17:31:23.040315 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:23.040285 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:28.482978 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.482945 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:28.483546 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.483210 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" podUID="e4018913-c621-48df-b46b-67a0d8f14fc6" containerName="maas-api" containerID="cri-o://341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674" gracePeriod=30 Jul 02 17:31:28.727337 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.727315 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:28.807736 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.807651 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca\") pod \"e4018913-c621-48df-b46b-67a0d8f14fc6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " Jul 02 17:31:28.807736 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.807707 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vw6w\" (UniqueName: \"kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w\") pod \"e4018913-c621-48df-b46b-67a0d8f14fc6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " Jul 02 17:31:28.807921 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.807748 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") pod \"e4018913-c621-48df-b46b-67a0d8f14fc6\" (UID: \"e4018913-c621-48df-b46b-67a0d8f14fc6\") " Jul 02 17:31:28.808037 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.808014 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "e4018913-c621-48df-b46b-67a0d8f14fc6" (UID: "e4018913-c621-48df-b46b-67a0d8f14fc6"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:28.809918 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.809883 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "e4018913-c621-48df-b46b-67a0d8f14fc6" (UID: "e4018913-c621-48df-b46b-67a0d8f14fc6"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:31:28.810031 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.809939 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w" (OuterVolumeSpecName: "kube-api-access-9vw6w") pod "e4018913-c621-48df-b46b-67a0d8f14fc6" (UID: "e4018913-c621-48df-b46b-67a0d8f14fc6"). InnerVolumeSpecName "kube-api-access-9vw6w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:28.909010 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.908974 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/e4018913-c621-48df-b46b-67a0d8f14fc6-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:28.909010 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.909002 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vw6w\" (UniqueName: \"kubernetes.io/projected/e4018913-c621-48df-b46b-67a0d8f14fc6-kube-api-access-9vw6w\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:28.909010 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:28.909012 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/e4018913-c621-48df-b46b-67a0d8f14fc6-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:29.089762 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.089672 2565 generic.go:356] "Generic (PLEG): container finished" podID="e4018913-c621-48df-b46b-67a0d8f14fc6" containerID="341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674" exitCode=0 Jul 02 17:31:29.089762 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.089721 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" Jul 02 17:31:29.089762 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.089751 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" event={"ID":"e4018913-c621-48df-b46b-67a0d8f14fc6","Type":"ContainerDied","Data":"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674"} Jul 02 17:31:29.090012 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.089807 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6" event={"ID":"e4018913-c621-48df-b46b-67a0d8f14fc6","Type":"ContainerDied","Data":"510b2c37428e02a182a3be7abccbfbd676e8cb5889fd2e7233b230a477bd5458"} Jul 02 17:31:29.090012 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.089828 2565 scope.go:117] "RemoveContainer" containerID="341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674" Jul 02 17:31:29.099429 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.099410 2565 scope.go:117] "RemoveContainer" containerID="341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674" Jul 02 17:31:29.099675 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:29.099656 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674\": container with ID starting with 341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674 not found: ID does not exist" containerID="341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674" Jul 02 17:31:29.099722 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.099684 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674"} err="failed to get container status \"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674\": rpc error: code = NotFound desc = could not find container \"341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674\": container with ID starting with 341a86c3a970c34ad986364e073023d626e96ee6f30f2eed170624af38876674 not found: ID does not exist" Jul 02 17:31:29.110527 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.110497 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:29.115633 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:29.115613 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-a-28df41-6c479485f5-rzkk6"] Jul 02 17:31:30.494318 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:30.494277 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="e4018913-c621-48df-b46b-67a0d8f14fc6" path="/var/lib/kubelet/pods/e4018913-c621-48df-b46b-67a0d8f14fc6/volumes" Jul 02 17:31:36.179427 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.179395 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:36.179961 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.179613 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" podUID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" containerName="maas-api" containerID="cri-o://a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce" gracePeriod=30 Jul 02 17:31:36.428434 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.428412 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:36.487830 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.487757 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca\") pod \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " Jul 02 17:31:36.487978 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.487837 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls\") pod \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " Jul 02 17:31:36.487978 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.487931 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mrck7\" (UniqueName: \"kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7\") pod \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\" (UID: \"62b84d32-f2a0-4a5d-898b-05f9e96cb39e\") " Jul 02 17:31:36.488158 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.488108 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "62b84d32-f2a0-4a5d-898b-05f9e96cb39e" (UID: "62b84d32-f2a0-4a5d-898b-05f9e96cb39e"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:31:36.488296 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.488280 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:36.489989 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.489970 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "62b84d32-f2a0-4a5d-898b-05f9e96cb39e" (UID: "62b84d32-f2a0-4a5d-898b-05f9e96cb39e"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:31:36.490059 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.490014 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7" (OuterVolumeSpecName: "kube-api-access-mrck7") pod "62b84d32-f2a0-4a5d-898b-05f9e96cb39e" (UID: "62b84d32-f2a0-4a5d-898b-05f9e96cb39e"). InnerVolumeSpecName "kube-api-access-mrck7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:31:36.589212 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.589187 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:36.589212 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:36.589210 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mrck7\" (UniqueName: \"kubernetes.io/projected/62b84d32-f2a0-4a5d-898b-05f9e96cb39e-kube-api-access-mrck7\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:31:37.127216 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.127182 2565 generic.go:356] "Generic (PLEG): container finished" podID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" containerID="a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce" exitCode=0 Jul 02 17:31:37.127382 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.127240 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" Jul 02 17:31:37.127382 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.127264 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" event={"ID":"62b84d32-f2a0-4a5d-898b-05f9e96cb39e","Type":"ContainerDied","Data":"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce"} Jul 02 17:31:37.127382 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.127327 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p" event={"ID":"62b84d32-f2a0-4a5d-898b-05f9e96cb39e","Type":"ContainerDied","Data":"75bb4b989f11812e5a618e92e5b2689c73f41b3dbc3d18485afe55bebc886543"} Jul 02 17:31:37.127382 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.127351 2565 scope.go:117] "RemoveContainer" containerID="a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce" Jul 02 17:31:37.137929 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.137912 2565 scope.go:117] "RemoveContainer" containerID="a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce" Jul 02 17:31:37.138258 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:37.138230 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce\": container with ID starting with a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce not found: ID does not exist" containerID="a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce" Jul 02 17:31:37.138356 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.138271 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce"} err="failed to get container status \"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce\": rpc error: code = NotFound desc = could not find container \"a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce\": container with ID starting with a7fa5156020bdb22ede9a190d7c87cbe5d4832bc255477ded584d37997f5a5ce not found: ID does not exist" Jul 02 17:31:37.142712 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.142690 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:37.146519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:37.146500 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-api-b-a295d5-fb4497f77-4978p"] Jul 02 17:31:38.494024 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:38.493992 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" path="/var/lib/kubelet/pods/62b84d32-f2a0-4a5d-898b-05f9e96cb39e/volumes" Jul 02 17:31:46.104827 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.104792 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105159 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" containerName="maas-api" Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105171 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" containerName="maas-api" Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105181 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e4018913-c621-48df-b46b-67a0d8f14fc6" containerName="maas-api" Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105186 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="e4018913-c621-48df-b46b-67a0d8f14fc6" containerName="maas-api" Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105247 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="e4018913-c621-48df-b46b-67a0d8f14fc6" containerName="maas-api" Jul 02 17:31:46.105254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.105256 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="62b84d32-f2a0-4a5d-898b-05f9e96cb39e" containerName="maas-api" Jul 02 17:31:46.110149 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.110101 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.112985 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.112958 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-a-a0a2ac-serving-cert\"" Jul 02 17:31:46.117192 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.117169 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:31:46.281001 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.280967 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.281231 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.281047 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnfk2\" (UniqueName: \"kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.281231 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.281081 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.382259 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.382172 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.382426 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.382289 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.382426 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:46.382331 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-a-a0a2ac-serving-cert: secret "maas-api-e2e-shared-a-a0a2ac-serving-cert" not found Jul 02 17:31:46.382426 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.382355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnfk2\" (UniqueName: \"kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.382426 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:46.382412 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls podName:baea534e-ff81-43cb-b910-4e46ce5ea2b9 nodeName:}" failed. No retries permitted until 2026-07-02 17:31:46.882390404 +0000 UTC m=+2234.842739478 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls") pod "maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" (UID: "baea534e-ff81-43cb-b910-4e46ce5ea2b9") : secret "maas-api-e2e-shared-a-a0a2ac-serving-cert" not found Jul 02 17:31:46.382967 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.382941 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.391814 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.391782 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnfk2\" (UniqueName: \"kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.887976 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.887943 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:46.890477 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:46.890447 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") pod \"maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:47.023910 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:47.023873 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:47.163436 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:47.163408 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:31:47.165396 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:31:47.165366 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbaea534e_ff81_43cb_b910_4e46ce5ea2b9.slice/crio-51045e4b780f2ce600dedc452c313e352335e06605cac8188d8ad7e23afb9a12 WatchSource:0}: Error finding container 51045e4b780f2ce600dedc452c313e352335e06605cac8188d8ad7e23afb9a12: Status 404 returned error can't find the container with id 51045e4b780f2ce600dedc452c313e352335e06605cac8188d8ad7e23afb9a12 Jul 02 17:31:47.178006 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:47.177979 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" event={"ID":"baea534e-ff81-43cb-b910-4e46ce5ea2b9","Type":"ContainerStarted","Data":"51045e4b780f2ce600dedc452c313e352335e06605cac8188d8ad7e23afb9a12"} Jul 02 17:31:48.183794 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:48.183753 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" event={"ID":"baea534e-ff81-43cb-b910-4e46ce5ea2b9","Type":"ContainerStarted","Data":"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543"} Jul 02 17:31:48.184292 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:48.183827 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:48.199092 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:48.199046 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" podStartSLOduration=1.881316435 podStartE2EDuration="2.199026276s" totalImagesPullingTime="317.709841ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:48.198931073 +0000 UTC m=+2236.159280149" watchObservedRunningTime="2026-07-02 17:31:48.199026276 +0000 UTC m=+2236.159375354" Jul 02 17:31:54.194074 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.193993 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:31:54.272686 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.272651 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:31:54.276278 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.276254 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.278679 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.278656 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-e2e-shared-b-4421aa-serving-cert\"" Jul 02 17:31:54.287436 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.287412 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:31:54.357071 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.357039 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.357071 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.357071 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.357287 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.357198 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hxrpf\" (UniqueName: \"kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.458426 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.458355 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.458426 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.458387 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.458607 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.458438 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hxrpf\" (UniqueName: \"kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.458607 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:54.458488 2565 secret.go:189] Couldn't get secret opendatahub/maas-api-e2e-shared-b-4421aa-serving-cert: secret "maas-api-e2e-shared-b-4421aa-serving-cert" not found Jul 02 17:31:54.458607 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:31:54.458555 2565 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls podName:a99fa7e5-16d9-478c-a29c-00e415ccaca1 nodeName:}" failed. No retries permitted until 2026-07-02 17:31:54.958535786 +0000 UTC m=+2242.918884839 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls") pod "maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" (UID: "a99fa7e5-16d9-478c-a29c-00e415ccaca1") : secret "maas-api-e2e-shared-b-4421aa-serving-cert" not found Jul 02 17:31:54.459035 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.459018 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.467854 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.467829 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hxrpf\" (UniqueName: \"kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.962854 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.962816 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:54.965309 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:54.965291 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") pod \"maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:55.187532 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:55.187498 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:55.321728 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:55.321705 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:31:55.323850 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:31:55.323821 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99fa7e5_16d9_478c_a29c_00e415ccaca1.slice/crio-0c448dfe01596d8ea1e3c8099f0adcde6e8c03701a88d3501bfa3db51e35b800 WatchSource:0}: Error finding container 0c448dfe01596d8ea1e3c8099f0adcde6e8c03701a88d3501bfa3db51e35b800: Status 404 returned error can't find the container with id 0c448dfe01596d8ea1e3c8099f0adcde6e8c03701a88d3501bfa3db51e35b800 Jul 02 17:31:56.221754 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:56.221716 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" event={"ID":"a99fa7e5-16d9-478c-a29c-00e415ccaca1","Type":"ContainerStarted","Data":"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215"} Jul 02 17:31:56.221916 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:56.221773 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" event={"ID":"a99fa7e5-16d9-478c-a29c-00e415ccaca1","Type":"ContainerStarted","Data":"0c448dfe01596d8ea1e3c8099f0adcde6e8c03701a88d3501bfa3db51e35b800"} Jul 02 17:31:56.221916 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:56.221804 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:31:56.238097 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:31:56.238049 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" podStartSLOduration=1.887942198 podStartE2EDuration="2.238033437s" totalImagesPullingTime="350.091239ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:31:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:31:56.236692804 +0000 UTC m=+2244.197041879" watchObservedRunningTime="2026-07-02 17:31:56.238033437 +0000 UTC m=+2244.198382513" Jul 02 17:32:02.231898 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:32:02.231866 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:34:32.658917 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:34:32.658891 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:34:32.667091 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:34:32.667069 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-bzchs_046db9f0-94ed-46b7-ba2c-b80bc25212a4/ovn-acl-logging/0.log" Jul 02 17:35:58.561668 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.561625 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:35:58.565092 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.565073 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.567784 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.567762 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a0a2ac\"/\"kube-root-ca.crt\"" Jul 02 17:35:58.567874 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.567818 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a0a2ac\"/\"openshift-service-ca.crt\"" Jul 02 17:35:58.568779 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.568762 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a0a2ac\"/\"rate-test-model-a0a2ac-kserve-self-signed-certs\"" Jul 02 17:35:58.568779 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.568777 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a0a2ac\"/\"default-dockercfg-sxsf6\"" Jul 02 17:35:58.573471 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.573444 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:35:58.677548 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677519 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.677548 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677553 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8w9rk\" (UniqueName: \"kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.677775 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677593 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.677775 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677634 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.677775 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677762 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.677880 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.677792 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778505 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778463 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778505 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778506 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778756 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778560 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778756 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778581 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-8w9rk\" (UniqueName: \"kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778756 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778618 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778756 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778632 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778952 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778915 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.778952 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.778951 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.779054 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.779024 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.781073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.781048 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.781409 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.781387 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.793702 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.793670 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8w9rk\" (UniqueName: \"kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk\") pod \"rate-test-model-a0a2ac-kserve-57965989d7-dxfkf\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:58.874918 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:58.874806 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:35:59.014907 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:59.014880 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:35:59.017093 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:35:59.017066 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbf4ba2f_774e_4b71_be1d_b97f9a0873dc.slice/crio-6fd6eade79a7333989d1ba5aea2e643ae1f16893a501017cbce589174ec9964e WatchSource:0}: Error finding container 6fd6eade79a7333989d1ba5aea2e643ae1f16893a501017cbce589174ec9964e: Status 404 returned error can't find the container with id 6fd6eade79a7333989d1ba5aea2e643ae1f16893a501017cbce589174ec9964e Jul 02 17:35:59.356758 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:59.356725 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" event={"ID":"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc","Type":"ContainerStarted","Data":"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14"} Jul 02 17:35:59.356941 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:59.356768 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" event={"ID":"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc","Type":"ContainerStarted","Data":"6fd6eade79a7333989d1ba5aea2e643ae1f16893a501017cbce589174ec9964e"} Jul 02 17:35:59.372439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:35:59.372391 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" podStartSLOduration=1.3723708559999999 podStartE2EDuration="1.372370856s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-02 17:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:35:59.371331508 +0000 UTC m=+2487.331680585" watchObservedRunningTime="2026-07-02 17:35:59.372370856 +0000 UTC m=+2487.332719931" Jul 02 17:36:08.875271 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:08.875233 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:36:08.875840 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:08.875308 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:36:08.887861 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:08.887832 2565 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:36:09.416012 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:09.415983 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:36:34.525531 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:34.525489 2565 scope.go:117] "RemoveContainer" containerID="ed9eec3a21a71915066b80074fcc55cd4184e5f37571930d2f19f117ada116ba" Jul 02 17:36:34.534231 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:36:34.534216 2565 scope.go:117] "RemoveContainer" containerID="f09a9b14eaa8b511c03686c5f2a8410e837ff3b1984098d95e4864ca2c0a15c2" Jul 02 17:37:25.867478 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:25.867441 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:37:28.878701 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:28.878662 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:37:28.879205 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:28.878919 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerName="maas-api" containerID="cri-o://bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543" gracePeriod=30 Jul 02 17:37:29.189547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.189509 2565 prober.go:120] "Probe failed" probeType="Readiness" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerName="maas-api" probeResult="failure" output="Get \"https://10.134.0.45:8443/health\": dial tcp 10.134.0.45:8443: connect: connection refused" Jul 02 17:37:29.329819 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.329796 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:37:29.449961 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.449929 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") pod \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " Jul 02 17:37:29.450158 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.450029 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca\") pod \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " Jul 02 17:37:29.450158 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.450051 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnfk2\" (UniqueName: \"kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2\") pod \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\" (UID: \"baea534e-ff81-43cb-b910-4e46ce5ea2b9\") " Jul 02 17:37:29.450449 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.450427 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "baea534e-ff81-43cb-b910-4e46ce5ea2b9" (UID: "baea534e-ff81-43cb-b910-4e46ce5ea2b9"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:37:29.452350 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.452327 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "baea534e-ff81-43cb-b910-4e46ce5ea2b9" (UID: "baea534e-ff81-43cb-b910-4e46ce5ea2b9"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:37:29.452412 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.452367 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2" (OuterVolumeSpecName: "kube-api-access-hnfk2") pod "baea534e-ff81-43cb-b910-4e46ce5ea2b9" (UID: "baea534e-ff81-43cb-b910-4e46ce5ea2b9"). InnerVolumeSpecName "kube-api-access-hnfk2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:37:29.550958 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.550911 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/baea534e-ff81-43cb-b910-4e46ce5ea2b9-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:29.550958 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.550956 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnfk2\" (UniqueName: \"kubernetes.io/projected/baea534e-ff81-43cb-b910-4e46ce5ea2b9-kube-api-access-hnfk2\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:29.550958 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.550968 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/baea534e-ff81-43cb-b910-4e46ce5ea2b9-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:29.777884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.777796 2565 generic.go:356] "Generic (PLEG): container finished" podID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerID="bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543" exitCode=0 Jul 02 17:37:29.777884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.777841 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" event={"ID":"baea534e-ff81-43cb-b910-4e46ce5ea2b9","Type":"ContainerDied","Data":"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543"} Jul 02 17:37:29.777884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.777885 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" event={"ID":"baea534e-ff81-43cb-b910-4e46ce5ea2b9","Type":"ContainerDied","Data":"51045e4b780f2ce600dedc452c313e352335e06605cac8188d8ad7e23afb9a12"} Jul 02 17:37:29.778106 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.777906 2565 scope.go:117] "RemoveContainer" containerID="bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543" Jul 02 17:37:29.778106 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.777848 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff" Jul 02 17:37:29.787200 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.787177 2565 scope.go:117] "RemoveContainer" containerID="bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543" Jul 02 17:37:29.787471 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:37:29.787450 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543\": container with ID starting with bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543 not found: ID does not exist" containerID="bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543" Jul 02 17:37:29.787546 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.787481 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543"} err="failed to get container status \"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543\": rpc error: code = NotFound desc = could not find container \"bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543\": container with ID starting with bc77eef52f07f6c23d6d265ba8f2e97a0c4c8d90d75590e99ae6ebffa19ee543 not found: ID does not exist" Jul 02 17:37:29.797833 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.797808 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:37:29.803959 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:29.803938 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-a-a0a2ac-bd4cbc96c-h94ff"] Jul 02 17:37:30.494297 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:30.494264 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" path="/var/lib/kubelet/pods/baea534e-ff81-43cb-b910-4e46ce5ea2b9/volumes" Jul 02 17:37:40.887873 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:40.887807 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" podUID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" containerName="main" containerID="cri-o://28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14" gracePeriod=45 Jul 02 17:37:41.135513 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.135488 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:37:41.152875 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.152646 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153223 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153179 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache" (OuterVolumeSpecName: "model-cache") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153383 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153428 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8w9rk\" (UniqueName: \"kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153465 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153490 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153518 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir\") pod \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\" (UID: \"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc\") " Jul 02 17:37:41.153993 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.153814 2565 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-model-cache\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.154464 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.154432 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:37:41.154679 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.154639 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home" (OuterVolumeSpecName: "home") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:37:41.157394 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.157358 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:37:41.157688 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.157642 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm" (OuterVolumeSpecName: "dshm") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 02 17:37:41.157772 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.157727 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk" (OuterVolumeSpecName: "kube-api-access-8w9rk") pod "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" (UID: "bbf4ba2f-774e-4b71-be1d-b97f9a0873dc"). InnerVolumeSpecName "kube-api-access-8w9rk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:37:41.255179 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.255113 2565 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-home\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.255179 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.255174 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8w9rk\" (UniqueName: \"kubernetes.io/projected/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-kube-api-access-8w9rk\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.255179 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.255188 2565 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tls-certs\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.255425 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.255198 2565 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-dshm\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.255425 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.255205 2565 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc-tmp-dir\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:37:41.842191 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.842157 2565 generic.go:356] "Generic (PLEG): container finished" podID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" containerID="28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14" exitCode=0 Jul 02 17:37:41.842370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.842205 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" Jul 02 17:37:41.842370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.842223 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" event={"ID":"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc","Type":"ContainerDied","Data":"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14"} Jul 02 17:37:41.842370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.842265 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf" event={"ID":"bbf4ba2f-774e-4b71-be1d-b97f9a0873dc","Type":"ContainerDied","Data":"6fd6eade79a7333989d1ba5aea2e643ae1f16893a501017cbce589174ec9964e"} Jul 02 17:37:41.842370 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.842283 2565 scope.go:117] "RemoveContainer" containerID="28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14" Jul 02 17:37:41.851530 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.851511 2565 scope.go:117] "RemoveContainer" containerID="28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14" Jul 02 17:37:41.851817 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:37:41.851795 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14\": container with ID starting with 28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14 not found: ID does not exist" containerID="28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14" Jul 02 17:37:41.851884 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.851830 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14"} err="failed to get container status \"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14\": rpc error: code = NotFound desc = could not find container \"28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14\": container with ID starting with 28fdd887966582e80e7b1ad9c21dac6acbe4fcf559e94e6d638431fa53381f14 not found: ID does not exist" Jul 02 17:37:41.863887 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.863857 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:37:41.867017 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:41.866995 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a0a2ac/rate-test-model-a0a2ac-kserve-57965989d7-dxfkf"] Jul 02 17:37:42.493994 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:37:42.493959 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" path="/var/lib/kubelet/pods/bbf4ba2f-774e-4b71-be1d-b97f9a0873dc/volumes" Jul 02 17:38:11.864158 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:11.864099 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:38:11.864710 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:11.864404 2565 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" podUID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" containerName="maas-api" containerID="cri-o://abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215" gracePeriod=30 Jul 02 17:38:12.121879 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.121813 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:38:12.239853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.239816 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca\") pod \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " Jul 02 17:38:12.239853 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.239860 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hxrpf\" (UniqueName: \"kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf\") pod \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " Jul 02 17:38:12.240087 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.239938 2565 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") pod \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\" (UID: \"a99fa7e5-16d9-478c-a29c-00e415ccaca1\") " Jul 02 17:38:12.240300 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.240274 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "a99fa7e5-16d9-478c-a29c-00e415ccaca1" (UID: "a99fa7e5-16d9-478c-a29c-00e415ccaca1"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 02 17:38:12.242174 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.242145 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "a99fa7e5-16d9-478c-a29c-00e415ccaca1" (UID: "a99fa7e5-16d9-478c-a29c-00e415ccaca1"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 02 17:38:12.242333 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.242311 2565 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf" (OuterVolumeSpecName: "kube-api-access-hxrpf") pod "a99fa7e5-16d9-478c-a29c-00e415ccaca1" (UID: "a99fa7e5-16d9-478c-a29c-00e415ccaca1"). InnerVolumeSpecName "kube-api-access-hxrpf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 02 17:38:12.340828 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.340791 2565 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/a99fa7e5-16d9-478c-a29c-00e415ccaca1-openshift-service-ca\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:38:12.340828 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.340822 2565 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hxrpf\" (UniqueName: \"kubernetes.io/projected/a99fa7e5-16d9-478c-a29c-00e415ccaca1-kube-api-access-hxrpf\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:38:12.340828 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.340832 2565 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/a99fa7e5-16d9-478c-a29c-00e415ccaca1-maas-api-tls\") on node \"ip-10-0-130-120.ec2.internal\" DevicePath \"\"" Jul 02 17:38:12.989082 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.989052 2565 generic.go:356] "Generic (PLEG): container finished" podID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" containerID="abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215" exitCode=0 Jul 02 17:38:12.989519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.989101 2565 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" Jul 02 17:38:12.989519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.989145 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" event={"ID":"a99fa7e5-16d9-478c-a29c-00e415ccaca1","Type":"ContainerDied","Data":"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215"} Jul 02 17:38:12.989519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.989204 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj" event={"ID":"a99fa7e5-16d9-478c-a29c-00e415ccaca1","Type":"ContainerDied","Data":"0c448dfe01596d8ea1e3c8099f0adcde6e8c03701a88d3501bfa3db51e35b800"} Jul 02 17:38:12.989519 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.989226 2565 scope.go:117] "RemoveContainer" containerID="abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215" Jul 02 17:38:12.998256 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.998233 2565 scope.go:117] "RemoveContainer" containerID="abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215" Jul 02 17:38:12.998557 ip-10-0-130-120 kubenswrapper[2565]: E0702 17:38:12.998533 2565 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215\": container with ID starting with abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215 not found: ID does not exist" containerID="abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215" Jul 02 17:38:12.998628 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:12.998565 2565 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215"} err="failed to get container status \"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215\": rpc error: code = NotFound desc = could not find container \"abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215\": container with ID starting with abb7e50f3fbfcb03853e0e1b28f6bd67f07f4ca74c94c9c8a321fa3ad1b0c215 not found: ID does not exist" Jul 02 17:38:13.005493 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:13.005464 2565 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:38:13.008803 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:13.008777 2565 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-api-e2e-shared-b-4421aa-d467c5cb9-vd7gj"] Jul 02 17:38:14.494476 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:14.494442 2565 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" path="/var/lib/kubelet/pods/a99fa7e5-16d9-478c-a29c-00e415ccaca1/volumes" Jul 02 17:38:23.720814 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:23.720768 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-647f5ff947-t7rql_fc000a1f-b533-484f-84cd-65679fa554e5/maas-api/0.log" Jul 02 17:38:23.830897 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:23.830863 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29716875-q84hw_c40c8da4-96f9-4b50-bc44-dda2762933a2/cleanup/0.log" Jul 02 17:38:23.958448 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:23.958417 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29716890-xdnvc_b1ad6447-7275-45e6-8267-c67e01e36d60/cleanup/0.log" Jul 02 17:38:24.088628 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:24.088545 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d749c4bcb-dff5l_2314b0b8-93c4-4e0d-82b6-f7ca614c33ee/manager/0.log" Jul 02 17:38:24.758394 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:24.758361 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-77zcl_901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7/copy-manifests/0.log" Jul 02 17:38:24.765602 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:24.765573 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-77zcl_901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7/manager/0.log" Jul 02 17:38:24.885356 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:24.885330 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-zg854_364d72ea-37f3-4160-a398-40e96f751a0e/postgres/0.log" Jul 02 17:38:27.713921 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:27.713886 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fzw2q_8736968b-ea34-41f0-842c-8eb181467684/discovery/0.log" Jul 02 17:38:28.770830 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:28.770798 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-bd989_db9e6493-f932-46b1-8dbd-041e9e712a58/main/0.log" Jul 02 17:38:28.884439 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:28.884404 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-dh2pk_930e5897-8b16-4d08-93dd-66896d73f483/main/0.log" Jul 02 17:38:40.079237 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:40.079198 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-qv2h5_b801cb53-4e0c-4a93-8a19-49f2a550187f/global-pull-secret-syncer/0.log" Jul 02 17:38:40.215292 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:40.215262 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-smpjh_91d5fcb7-2ac0-4139-8595-dcd0eef80101/konnectivity-agent/0.log" Jul 02 17:38:40.296641 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:40.296610 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-120.ec2.internal_404f9a43cfb35449789f47a2943d4dea/haproxy/0.log" Jul 02 17:38:47.950263 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:47.950232 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/alertmanager/0.log" Jul 02 17:38:47.975243 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:47.975217 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/config-reloader/0.log" Jul 02 17:38:47.998162 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:47.998135 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/kube-rbac-proxy-web/0.log" Jul 02 17:38:48.017949 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.017920 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/kube-rbac-proxy/0.log" Jul 02 17:38:48.054669 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.054643 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/kube-rbac-proxy-metric/0.log" Jul 02 17:38:48.076009 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.075981 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/prom-label-proxy/0.log" Jul 02 17:38:48.101086 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.101015 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_328c9f3b-e5eb-4713-93b7-69bc1d1597fd/init-config-reloader/0.log" Jul 02 17:38:48.404780 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.404711 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-77fccdd475-sgs6n_be009aff-7e76-468c-9094-c0cdfb883fdb/metrics-server/0.log" Jul 02 17:38:48.426545 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.426510 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-cd8656b8-wxrdt_6eaf70fc-f4c3-4b7b-9dbf-766744213ee6/monitoring-plugin/0.log" Jul 02 17:38:48.455842 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.455819 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6bx7_64bc2fc3-4c2a-426f-a801-2775c2892d9b/node-exporter/0.log" Jul 02 17:38:48.473969 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.473939 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6bx7_64bc2fc3-4c2a-426f-a801-2775c2892d9b/kube-rbac-proxy/0.log" Jul 02 17:38:48.497409 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.497388 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-k6bx7_64bc2fc3-4c2a-426f-a801-2775c2892d9b/init-textfile/0.log" Jul 02 17:38:48.752732 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.752703 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/prometheus/0.log" Jul 02 17:38:48.776254 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.776231 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/config-reloader/0.log" Jul 02 17:38:48.805437 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.805411 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/thanos-sidecar/0.log" Jul 02 17:38:48.828585 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.828563 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/kube-rbac-proxy-web/0.log" Jul 02 17:38:48.850750 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.850728 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/kube-rbac-proxy/0.log" Jul 02 17:38:48.873166 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.873109 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/kube-rbac-proxy-thanos/0.log" Jul 02 17:38:48.895618 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.895597 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_66f551bc-f528-4f10-aea5-8af4e1bbebd7/init-config-reloader/0.log" Jul 02 17:38:48.990877 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:48.990840 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b9fcd79d8-c5pxk_f70886d0-3e01-45fc-a5dc-837fa49e14aa/telemeter-client/0.log" Jul 02 17:38:49.014516 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.014422 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b9fcd79d8-c5pxk_f70886d0-3e01-45fc-a5dc-837fa49e14aa/reload/0.log" Jul 02 17:38:49.034555 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.034514 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7b9fcd79d8-c5pxk_f70886d0-3e01-45fc-a5dc-837fa49e14aa/kube-rbac-proxy/0.log" Jul 02 17:38:49.997562 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997524 2565 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r"] Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997880 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerName="maas-api" Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997891 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerName="maas-api" Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997900 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" containerName="maas-api" Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997906 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" containerName="maas-api" Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997918 2565 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" containerName="main" Jul 02 17:38:49.997974 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997924 2565 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" containerName="main" Jul 02 17:38:49.998224 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997983 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="baea534e-ff81-43cb-b910-4e46ce5ea2b9" containerName="maas-api" Jul 02 17:38:49.998224 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.997994 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="a99fa7e5-16d9-478c-a29c-00e415ccaca1" containerName="maas-api" Jul 02 17:38:49.998224 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:49.998001 2565 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbf4ba2f-774e-4b71-be1d-b97f9a0873dc" containerName="main" Jul 02 17:38:50.001247 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.001232 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.003814 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.003791 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tt4vl\"/\"kube-root-ca.crt\"" Jul 02 17:38:50.003953 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.003822 2565 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tt4vl\"/\"default-dockercfg-w8jbn\"" Jul 02 17:38:50.004766 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.004750 2565 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tt4vl\"/\"openshift-service-ca.crt\"" Jul 02 17:38:50.009202 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.009181 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r"] Jul 02 17:38:50.084448 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.084406 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tkh6m\" (UniqueName: \"kubernetes.io/projected/a342f274-5b8f-446b-b085-8bf255deab03-kube-api-access-tkh6m\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.084724 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.084460 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-proc\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.084724 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.084529 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-podres\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.084724 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.084637 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-lib-modules\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.084724 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.084668 2565 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-sys\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186173 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186107 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tkh6m\" (UniqueName: \"kubernetes.io/projected/a342f274-5b8f-446b-b085-8bf255deab03-kube-api-access-tkh6m\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186181 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-proc\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186206 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-podres\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186282 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-lib-modules\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186308 2565 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-sys\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186313 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-proc\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186395 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186370 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-sys\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186653 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186417 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-podres\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.186653 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.186428 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a342f274-5b8f-446b-b085-8bf255deab03-lib-modules\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.194707 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.194676 2565 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tkh6m\" (UniqueName: \"kubernetes.io/projected/a342f274-5b8f-446b-b085-8bf255deab03-kube-api-access-tkh6m\") pod \"perf-node-gather-daemonset-zsw7r\" (UID: \"a342f274-5b8f-446b-b085-8bf255deab03\") " pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.312718 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.312630 2565 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:50.444898 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:50.444865 2565 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r"] Jul 02 17:38:50.448472 ip-10-0-130-120 kubenswrapper[2565]: W0702 17:38:50.448446 2565 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda342f274_5b8f_446b_b085_8bf255deab03.slice/crio-e4757bfb35d02250150d30f7a3630f3e32a9c39b06d622e6de70487392ae1ac0 WatchSource:0}: Error finding container e4757bfb35d02250150d30f7a3630f3e32a9c39b06d622e6de70487392ae1ac0: Status 404 returned error can't find the container with id e4757bfb35d02250150d30f7a3630f3e32a9c39b06d622e6de70487392ae1ac0 Jul 02 17:38:51.162180 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:51.162108 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" event={"ID":"a342f274-5b8f-446b-b085-8bf255deab03","Type":"ContainerStarted","Data":"9dc6f07c7e5a8131e6358dec896f57f884b4693604e13530745f9f701d7de374"} Jul 02 17:38:51.162180 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:51.162182 2565 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" event={"ID":"a342f274-5b8f-446b-b085-8bf255deab03","Type":"ContainerStarted","Data":"e4757bfb35d02250150d30f7a3630f3e32a9c39b06d622e6de70487392ae1ac0"} Jul 02 17:38:51.162698 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:51.162302 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:51.178640 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:51.178586 2565 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" podStartSLOduration=2.178569752 podStartE2EDuration="2.178569752s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-02 17:38:49 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-02 17:38:51.176671049 +0000 UTC m=+2659.137020138" watchObservedRunningTime="2026-07-02 17:38:51.178569752 +0000 UTC m=+2659.138918828" Jul 02 17:38:53.044980 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:53.044949 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfmwc_ad223535-bdc2-46e6-aa2a-f56fff74dde0/dns/0.log" Jul 02 17:38:53.082813 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:53.082785 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gfmwc_ad223535-bdc2-46e6-aa2a-f56fff74dde0/kube-rbac-proxy/0.log" Jul 02 17:38:53.249862 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:53.249832 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-rqqbn_25d7ffcd-d7bd-4be9-9042-4ca1d3d00e9d/dns-node-resolver/0.log" Jul 02 17:38:53.860004 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:53.859973 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-d2hr6_4e80c9d4-a76a-46f4-9554-c258290c37b4/node-ca/0.log" Jul 02 17:38:55.001547 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:55.001518 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fzw2q_8736968b-ea34-41f0-842c-8eb181467684/discovery/0.log" Jul 02 17:38:55.959612 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:55.959583 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-bzpjp_fd042de0-a4db-45c2-b7b4-745fa4f7f186/serve-healthcheck-canary/0.log" Jul 02 17:38:56.600528 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:56.600492 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflwq_b03c864f-be23-47d2-bfbb-4e7113473a11/kube-rbac-proxy/0.log" Jul 02 17:38:56.628073 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:56.628037 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflwq_b03c864f-be23-47d2-bfbb-4e7113473a11/exporter/0.log" Jul 02 17:38:56.650542 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:56.650519 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-qflwq_b03c864f-be23-47d2-bfbb-4e7113473a11/extractor/0.log" Jul 02 17:38:57.177835 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:57.177810 2565 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-tt4vl/perf-node-gather-daemonset-zsw7r" Jul 02 17:38:59.080767 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.080734 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-647f5ff947-t7rql_fc000a1f-b533-484f-84cd-65679fa554e5/maas-api/0.log" Jul 02 17:38:59.101766 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.101739 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29716875-q84hw_c40c8da4-96f9-4b50-bc44-dda2762933a2/cleanup/0.log" Jul 02 17:38:59.123168 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.123147 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-key-cleanup-29716890-xdnvc_b1ad6447-7275-45e6-8267-c67e01e36d60/cleanup/0.log" Jul 02 17:38:59.290263 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.290233 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5d749c4bcb-dff5l_2314b0b8-93c4-4e0d-82b6-f7ca614c33ee/manager/0.log" Jul 02 17:38:59.630090 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.630060 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-77zcl_901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7/manager/0.log" Jul 02 17:38:59.650713 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.650687 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-845f459c56-77zcl_901607d1-a9fb-4d5e-82a4-55e2ec5bc3b7/copy-manifests/0.log" Jul 02 17:38:59.677140 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:38:59.677098 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-69fb4594cf-zg854_364d72ea-37f3-4160-a398-40e96f751a0e/postgres/0.log" Jul 02 17:39:00.492822 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:00.492790 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fzw2q_8736968b-ea34-41f0-842c-8eb181467684/discovery/0.log" Jul 02 17:39:01.207008 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:01.206979 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-f87dff94c-qv99b_7dc964fe-c479-4193-94d7-7e35cf11db6d/manager/0.log" Jul 02 17:39:02.110817 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:02.110791 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-fzw2q_8736968b-ea34-41f0-842c-8eb181467684/discovery/0.log" Jul 02 17:39:06.362140 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:06.362091 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-bs4x7_d350d8d6-949d-4f32-9bdc-33f6070cb5c3/kube-storage-version-migrator-operator/1.log" Jul 02 17:39:06.363698 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:06.363676 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-79964ff6f9-bs4x7_d350d8d6-949d-4f32-9bdc-33f6070cb5c3/kube-storage-version-migrator-operator/0.log" Jul 02 17:39:07.684393 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.684368 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/kube-multus-additional-cni-plugins/0.log" Jul 02 17:39:07.709068 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.709032 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/egress-router-binary-copy/0.log" Jul 02 17:39:07.732308 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.732285 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/cni-plugins/0.log" Jul 02 17:39:07.754856 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.754837 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/bond-cni-plugin/0.log" Jul 02 17:39:07.776508 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.776482 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/routeoverride-cni/0.log" Jul 02 17:39:07.797911 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.797887 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/whereabouts-cni-bincopy/0.log" Jul 02 17:39:07.820376 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.820354 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w29hl_14f5451f-a4c8-4cc2-9457-4e12e2ccb347/whereabouts-cni/0.log" Jul 02 17:39:07.858464 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:07.858435 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-ld74z_02724429-317a-4f29-a028-2235b50f3cdb/kube-multus/0.log" Jul 02 17:39:08.016282 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:08.016202 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g7wz9_863c0cda-2dc4-467b-bf4d-5f8ef80a90a9/network-metrics-daemon/0.log" Jul 02 17:39:08.073688 ip-10-0-130-120 kubenswrapper[2565]: I0702 17:39:08.073655 2565 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-g7wz9_863c0cda-2dc4-467b-bf4d-5f8ef80a90a9/kube-rbac-proxy/0.log"