Jul 11 23:31:20.129763 ip-10-0-140-223 systemd[1]: Starting Kubernetes Kubelet... Jul 11 23:31:20.555481 ip-10-0-140-223 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 23:31:20.555481 ip-10-0-140-223 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 11 23:31:20.555481 ip-10-0-140-223 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 23:31:20.555481 ip-10-0-140-223 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 11 23:31:20.555481 ip-10-0-140-223 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 11 23:31:20.556613 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.556449 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 11 23:31:20.564230 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564212 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.564230 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564229 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564234 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564238 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564241 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564244 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564248 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564251 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564254 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564257 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564260 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564263 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564267 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564269 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564272 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564276 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564279 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564282 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564285 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564287 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564290 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.564305 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564294 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564297 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564299 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564302 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564305 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564309 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564312 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564315 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564318 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564321 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564324 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564329 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564333 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564336 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564339 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564342 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564345 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564348 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564351 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564353 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.564809 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564356 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564360 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564362 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564365 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564368 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564371 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564374 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564377 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564379 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564382 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564387 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564389 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564392 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564395 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564399 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564402 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564405 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564408 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564411 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564414 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.565320 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564416 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564419 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564422 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564425 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564427 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564432 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564436 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564439 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564442 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564445 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564448 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564451 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564454 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564457 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564459 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564462 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564465 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564468 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564470 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.565856 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564473 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564476 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564479 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564482 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564486 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564488 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564491 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564496 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564499 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564502 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564505 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564507 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564512 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564515 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564519 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564522 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564525 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564528 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.564531 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565293 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.566331 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565300 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565304 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565307 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565311 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565314 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565317 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565320 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565323 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565328 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565333 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565336 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565339 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565342 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565346 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565349 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565352 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565355 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565359 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565362 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565366 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.566836 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565369 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565372 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565375 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565377 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565380 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565383 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565386 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565388 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565391 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565393 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565396 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565399 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565401 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565404 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565407 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565409 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565412 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565415 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565418 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.567380 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565421 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565424 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565426 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565429 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565432 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565435 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565437 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565440 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565443 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565446 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565449 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565452 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565455 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565458 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565461 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565464 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565467 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565471 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565475 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.567889 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565478 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565481 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565484 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565487 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565490 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565493 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565495 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565498 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565501 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565503 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565506 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565508 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565512 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565515 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565517 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565520 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565523 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565526 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565529 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565532 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.568395 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565535 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565537 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565540 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565559 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565563 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565566 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565570 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565573 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565576 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565579 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565582 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565585 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565588 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565590 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565593 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565596 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565599 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565601 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565606 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.568924 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.565609 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566001 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566007 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566011 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566015 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566019 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566022 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566028 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566031 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566035 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566039 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566042 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566045 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566048 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566050 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566053 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566056 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566059 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566062 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.569454 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566065 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566068 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566070 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566073 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566076 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566079 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566081 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566084 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566087 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566089 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566092 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566095 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566098 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566100 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566103 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566106 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566108 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566111 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566113 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566118 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.569988 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566122 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566125 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566128 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566130 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566133 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566136 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566138 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566141 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566145 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566148 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566151 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566154 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566156 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566159 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566162 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566165 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566167 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566170 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566173 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.570496 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566176 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566178 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566181 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566184 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566186 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566189 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566192 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566194 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566197 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566200 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566202 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566205 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566208 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566212 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566215 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566218 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566220 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566223 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566226 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566228 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.571040 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566231 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566236 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566239 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566242 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566244 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566247 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566250 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566252 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566255 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566258 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566261 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566264 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566267 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566271 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566274 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566277 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566280 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566284 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566287 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566290 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.571540 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566293 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.566295 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566373 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566385 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566391 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566397 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566404 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566407 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566412 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566416 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566420 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566423 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566426 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566431 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566435 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566438 2580 flags.go:64] FLAG: --cgroup-root="" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566441 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566444 2580 flags.go:64] FLAG: --client-ca-file="" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566447 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566451 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566455 2580 flags.go:64] FLAG: --cluster-domain="" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566458 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566461 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566465 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 11 23:31:20.572076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566468 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566472 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566475 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566478 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566482 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566485 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566488 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566491 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566494 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566497 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566502 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566505 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566508 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566515 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566518 2580 flags.go:64] FLAG: --enable-server="true" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566522 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566526 2580 flags.go:64] FLAG: --event-burst="100" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566529 2580 flags.go:64] FLAG: --event-qps="50" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566532 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566535 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566539 2580 flags.go:64] FLAG: --eviction-hard="" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566560 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566563 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566566 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566570 2580 flags.go:64] FLAG: --eviction-soft="" Jul 11 23:31:20.572762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566573 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566576 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566580 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566583 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566586 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566589 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566592 2580 flags.go:64] FLAG: --feature-gates="" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566596 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566599 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566602 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566605 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566608 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566612 2580 flags.go:64] FLAG: --help="false" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566615 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566618 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566621 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566624 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566627 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566631 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566634 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566638 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566641 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566644 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 11 23:31:20.573418 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566648 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566651 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566654 2580 flags.go:64] FLAG: --kube-reserved="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566657 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566660 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566665 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566668 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566671 2580 flags.go:64] FLAG: --lock-file="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566674 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566677 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566681 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566686 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566689 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566692 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566695 2580 flags.go:64] FLAG: --logging-format="text" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566698 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566702 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566705 2580 flags.go:64] FLAG: --manifest-url="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566708 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566712 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566716 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566720 2580 flags.go:64] FLAG: --max-pods="110" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566723 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566726 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566729 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 11 23:31:20.574073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566732 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566735 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566738 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566741 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566750 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566753 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566757 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566760 2580 flags.go:64] FLAG: --pod-cidr="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566764 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566770 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566773 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566777 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566781 2580 flags.go:64] FLAG: --port="10250" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566784 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566788 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-02b94db7718d76273" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566791 2580 flags.go:64] FLAG: --qos-reserved="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566794 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566797 2580 flags.go:64] FLAG: --register-node="true" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566800 2580 flags.go:64] FLAG: --register-with-taints="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566804 2580 flags.go:64] FLAG: --registry-burst="10" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566807 2580 flags.go:64] FLAG: --registry-qps="5" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566810 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566813 2580 flags.go:64] FLAG: --reserved-memory="" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566816 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566820 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 11 23:31:20.574747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566823 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566826 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566829 2580 flags.go:64] FLAG: --runonce="false" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566832 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566835 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566838 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566842 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566845 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566848 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566851 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566855 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566858 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566861 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566865 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566867 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566871 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566874 2580 flags.go:64] FLAG: --system-cgroups="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566877 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566883 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566886 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566889 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566894 2580 flags.go:64] FLAG: --tls-min-version="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566897 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566900 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566903 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 11 23:31:20.575396 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566906 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566909 2580 flags.go:64] FLAG: --v="2" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566913 2580 flags.go:64] FLAG: --version="false" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566917 2580 flags.go:64] FLAG: --vmodule="" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566921 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.566925 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567054 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567058 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567061 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567064 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567067 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567070 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567073 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567076 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567079 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567082 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567085 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567088 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567091 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567094 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567097 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.576063 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567100 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567102 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567105 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567108 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567111 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567114 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567117 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567120 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567123 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567125 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567128 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567131 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567134 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567137 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567139 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567142 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567144 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567148 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567150 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.576657 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567153 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567156 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567158 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567161 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567164 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567166 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567169 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567171 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567175 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567177 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567180 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567183 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567185 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567188 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567191 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567194 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567196 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567199 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567202 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567205 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.577154 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567208 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567211 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567213 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567216 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567219 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567221 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567224 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567227 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567230 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567232 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567235 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567237 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567242 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567245 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567249 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567251 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567254 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567257 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567260 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.577693 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567264 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567268 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567271 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567274 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567276 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567279 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567282 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567284 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567287 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567290 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567294 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567298 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567302 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567305 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567309 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567312 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567315 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567319 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567322 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567324 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.578178 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567327 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567330 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567333 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567336 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567339 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.567342 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.567351 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.574804 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.574824 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.574845 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.574852 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574924 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574931 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574936 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.578785 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574940 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574943 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574946 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574948 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574951 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574954 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574957 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574960 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574963 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574965 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574968 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574971 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574974 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574976 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574979 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574982 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574984 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574987 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574990 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574993 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.579159 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574996 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.574998 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575001 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575004 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575006 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575009 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575020 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575024 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575026 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575029 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575032 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575034 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575037 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575040 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575043 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575046 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575049 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575051 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575054 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.579685 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575057 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575060 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575062 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575065 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575068 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575070 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575073 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575076 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575078 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575081 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575084 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575086 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575089 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575092 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575094 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575097 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575100 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575102 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575106 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575108 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.580184 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575118 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575121 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575124 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575126 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575129 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575132 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575134 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575137 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575140 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575143 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575145 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575148 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575151 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575154 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575156 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575160 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575164 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575167 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575170 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575173 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.580716 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575176 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575180 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575183 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575186 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575189 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575192 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575195 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575198 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575201 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575204 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575209 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575214 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575217 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575226 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575229 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575232 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.581235 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575235 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.575241 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575407 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575413 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575416 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575419 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575423 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575427 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575429 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575432 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575435 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575438 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575440 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575443 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575446 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 11 23:31:20.581675 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575449 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575451 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575454 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575457 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575459 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575462 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575465 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575468 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575471 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575473 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575476 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575480 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575482 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575485 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575488 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575496 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575499 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575502 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575505 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575508 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 11 23:31:20.582064 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575510 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575513 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575516 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575519 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575521 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575524 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575526 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575529 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575532 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575535 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575537 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575540 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575556 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575559 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575561 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575564 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575567 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575570 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575573 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575575 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 11 23:31:20.582728 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575578 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575581 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575584 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575587 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575589 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575592 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575595 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575597 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575607 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575610 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575613 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575615 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575618 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575621 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575623 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575626 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575628 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575631 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575634 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575636 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 11 23:31:20.583294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575639 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575642 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575644 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575647 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575650 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575652 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575655 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575658 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575664 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575667 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575672 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575675 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575678 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575681 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575686 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575689 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575692 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575695 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575698 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 11 23:31:20.583813 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575703 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575706 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575715 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575718 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575721 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575724 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:20.575727 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.575732 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.575904 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.578092 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.579620 2580 server.go:1019] "Starting client certificate rotation" Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.579725 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.579756 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 11 23:31:20.584303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.583039 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 11 23:31:20.587151 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.587125 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 11 23:31:20.597919 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.597867 2580 log.go:25] "Validated CRI v1 runtime API" Jul 11 23:31:20.599984 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.599969 2580 log.go:25] "Validated CRI v1 image API" Jul 11 23:31:20.601474 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.601458 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 11 23:31:20.604459 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.604432 2580 fs.go:135] Filesystem UUIDs: map[60d10cb7-4bbf-48f0-8e9e-505b5228ad9e:/dev/nvme0n1p3 69c1b370-6e7f-44fb-9bec-98ae47bf5961:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2] Jul 11 23:31:20.604540 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.604457 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 11 23:31:20.607755 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.607722 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 11 23:31:20.609977 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.609868 2580 manager.go:216] Machine: {Timestamp:2026-07-11 23:31:20.608333783 +0000 UTC m=+0.360081280 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099053 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec20dc7608b42317702eb0ce06096ce5 SystemUUID:ec20dc76-08b4-2317-702e-b0ce06096ce5 BootID:10ad521e-5cb4-4931-9640-2569784d091d Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:39:0a:9a:99:39 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:39:0a:9a:99:39 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:9e:a7:90:36:e4:f5 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 11 23:31:20.609977 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.609969 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 11 23:31:20.610102 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.610049 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 11 23:31:20.613321 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.613296 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 11 23:31:20.613508 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.613325 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-140-223.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 11 23:31:20.614338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.614326 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 11 23:31:20.614409 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.614346 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 11 23:31:20.614409 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.614365 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 11 23:31:20.615199 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.615186 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 11 23:31:20.616478 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.616465 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 11 23:31:20.620619 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620604 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 11 23:31:20.620779 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620767 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 11 23:31:20.620842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620807 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 11 23:31:20.620842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620828 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 11 23:31:20.620940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620848 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 11 23:31:20.620940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620868 2580 kubelet.go:398] "Adding apiserver pod source" Jul 11 23:31:20.620940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.620881 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 11 23:31:20.621340 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.621325 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 11 23:31:20.621408 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.621349 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 11 23:31:20.623771 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.623752 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 11 23:31:20.625597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625581 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 11 23:31:20.625678 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625614 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 11 23:31:20.625794 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625783 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 11 23:31:20.625842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625799 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 11 23:31:20.625842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625806 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 11 23:31:20.625842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625812 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 11 23:31:20.625842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625821 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 11 23:31:20.625842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.625826 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 11 23:31:20.626642 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626631 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 11 23:31:20.626642 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626642 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626651 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626657 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626666 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626674 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626698 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 11 23:31:20.626702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.626704 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 11 23:31:20.629895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.629880 2580 server.go:1268] "Started kubelet" Jul 11 23:31:20.630255 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.630197 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 11 23:31:20.630331 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.630280 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 11 23:31:20.630509 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.630230 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 11 23:31:20.630804 ip-10-0-140-223 systemd[1]: Started Kubernetes Kubelet. Jul 11 23:31:20.634319 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.634291 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 11 23:31:20.636284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.636247 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mrcc8" Jul 11 23:31:20.636940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.636920 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-140-223.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 11 23:31:20.637037 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.636941 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 11 23:31:20.639919 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.639900 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 11 23:31:20.640893 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.640877 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 11 23:31:20.640893 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.640886 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 11 23:31:20.641139 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.641119 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:20.641532 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641504 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 11 23:31:20.641532 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641518 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 11 23:31:20.641532 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641534 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 11 23:31:20.641730 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641676 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 11 23:31:20.641730 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641685 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 11 23:31:20.641835 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641776 2580 factory.go:55] Registering systemd factory Jul 11 23:31:20.641835 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.641792 2580 factory.go:223] Registration of the systemd container factory successfully Jul 11 23:31:20.642046 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642017 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-mrcc8" Jul 11 23:31:20.642046 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642031 2580 factory.go:153] Registering CRI-O factory Jul 11 23:31:20.642198 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642073 2580 factory.go:223] Registration of the crio container factory successfully Jul 11 23:31:20.642198 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642133 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 11 23:31:20.642198 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642161 2580 factory.go:103] Registering Raw factory Jul 11 23:31:20.642198 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642176 2580 manager.go:1196] Started watching for new ooms in manager Jul 11 23:31:20.642414 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.642387 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-140-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 11 23:31:20.642646 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.642615 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 11 23:31:20.642852 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.642837 2580 manager.go:349] Starting recovery of all containers Jul 11 23:31:20.643743 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.642717 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-140-223.ec2.internal.18c1602e96c79ac5 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-140-223.ec2.internal,UID:ip-10-0-140-223.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-140-223.ec2.internal,},FirstTimestamp:2026-07-11 23:31:20.629852869 +0000 UTC m=+0.381600365,LastTimestamp:2026-07-11 23:31:20.629852869 +0000 UTC m=+0.381600365,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-140-223.ec2.internal,}" Jul 11 23:31:20.646641 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.646605 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-140-223.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 11 23:31:20.646741 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.646716 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 11 23:31:20.653250 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.653232 2580 manager.go:354] Recovery completed Jul 11 23:31:20.658466 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.658345 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.660358 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660340 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.660519 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660377 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.660519 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660392 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.660973 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660961 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 11 23:31:20.661032 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660974 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 11 23:31:20.661032 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660988 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.661032 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.660993 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 11 23:31:20.661747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.661733 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.661830 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.661762 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.661830 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.661777 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.663247 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.663234 2580 policy_none.go:49] "None policy: Start" Jul 11 23:31:20.663315 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.663252 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 11 23:31:20.663315 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.663266 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 11 23:31:20.665429 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.665414 2580 policy_none.go:47] "Start" Jul 11 23:31:20.697182 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697163 2580 manager.go:337] "Starting Device Plugin manager" Jul 11 23:31:20.697277 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.697200 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 11 23:31:20.697277 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697215 2580 server.go:87] "Starting device plugin registration server" Jul 11 23:31:20.697496 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697481 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 11 23:31:20.697533 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697500 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 11 23:31:20.697630 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697609 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 11 23:31:20.697752 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697707 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 11 23:31:20.697752 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.697722 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 11 23:31:20.698253 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.698228 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 11 23:31:20.698350 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.698280 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:20.780351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.780301 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 11 23:31:20.782690 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.781682 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 11 23:31:20.782690 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.781729 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 11 23:31:20.782690 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.781757 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 11 23:31:20.782690 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.781803 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 11 23:31:20.784251 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.784226 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:20.798006 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.797991 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.798870 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.798854 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.798940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.798891 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.798940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.798907 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.798940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.798937 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.806820 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.806804 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.806867 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.806827 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-140-223.ec2.internal\": node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:20.820999 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.820980 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:20.882806 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.882744 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"] Jul 11 23:31:20.882880 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.882813 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.884748 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.884729 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.884815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.884760 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.884815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.884770 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.887167 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.887154 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.887291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.887259 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.887291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.887290 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.888597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888582 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.888597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888588 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.888715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888611 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.888715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888613 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.888715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888623 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.888715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.888627 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.890893 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.890879 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.890944 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.890904 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 11 23:31:20.891680 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.891658 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientMemory" Jul 11 23:31:20.891772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.891687 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasNoDiskPressure" Jul 11 23:31:20.891772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.891698 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeHasSufficientPID" Jul 11 23:31:20.911453 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.911432 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-223.ec2.internal\" not found" node="ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.916084 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.916069 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-140-223.ec2.internal\" not found" node="ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.921786 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:20.921767 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:20.943127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.943109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.943216 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.943137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:20.943216 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:20.943157 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.021896 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.021870 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.043823 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.043892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043828 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.043892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043845 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.043963 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043909 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.043963 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043918 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/16a2c0a00cdbe9d77d0bff3f91f1c3e7-config\") pod \"kube-apiserver-proxy-ip-10-0-140-223.ec2.internal\" (UID: \"16a2c0a00cdbe9d77d0bff3f91f1c3e7\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.043963 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.043918 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/281d1b3a45b212ba9681d002edffe820-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal\" (UID: \"281d1b3a45b212ba9681d002edffe820\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.123531 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.123492 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.218190 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.218114 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.219296 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.219278 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.224493 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.224459 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.325583 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.325522 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.427333 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.427289 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.527673 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.527573 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-140-223.ec2.internal\" not found" Jul 11 23:31:21.574820 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.574790 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:21.578778 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.578763 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 11 23:31:21.578895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.578877 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 11 23:31:21.578948 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.578928 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 11 23:31:21.622689 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.622651 2580 apiserver.go:52] "Watching apiserver" Jul 11 23:31:21.640535 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.640511 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 11 23:31:21.640908 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.640884 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-2qvz4","openshift-multus/multus-88pmr","openshift-multus/network-metrics-daemon-cdt7s","openshift-network-diagnostics/network-check-target-b465v","openshift-ovn-kubernetes/ovnkube-node-dwvq7","kube-system/global-pull-secret-syncer-gstql","kube-system/konnectivity-agent-rw7vw","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc","openshift-image-registry/node-ca-m92gl","openshift-multus/multus-additional-cni-plugins-mbh6w","openshift-network-operator/iptables-alerter-8p8bs","openshift-cluster-node-tuning-operator/tuned-7k495"] Jul 11 23:31:21.641714 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.641690 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 11 23:31:21.641714 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.641703 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.643323 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.643307 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.645619 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.645598 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.645723 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.645680 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.645788 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.645729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.645788 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.645758 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-5nn7w\"" Jul 11 23:31:21.646852 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.646832 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b085832-ada8-407e-8fb8-ab9a7c310ee9-hosts-file\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.646927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.646870 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b085832-ada8-407e-8fb8-ab9a7c310ee9-tmp-dir\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.646927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.646896 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2fllx\" (UniqueName: \"kubernetes.io/projected/7b085832-ada8-407e-8fb8-ab9a7c310ee9-kube-api-access-2fllx\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.648719 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.646939 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-10 23:26:21 +0000 UTC" deadline="2028-03-09 02:00:20.079509458 +0000 UTC" Jul 11 23:31:21.648719 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.646970 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14546h28m58.432544478s" Jul 11 23:31:21.648919 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.648901 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.648984 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.648929 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.648984 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.648944 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.649154 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.648989 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:21.649154 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.649098 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 11 23:31:21.649921 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.649775 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-w2rkq\"" Jul 11 23:31:21.650093 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.649954 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 11 23:31:21.652308 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.652290 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:21.652413 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.652369 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:21.653433 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.653416 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 11 23:31:21.653534 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.653520 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" Jul 11 23:31:21.654835 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.654813 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.657152 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657131 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 11 23:31:21.657266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657176 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.657266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657250 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.657392 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.657321 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:21.657707 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657487 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 11 23:31:21.657707 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657600 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6gg67\"" Jul 11 23:31:21.657707 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657696 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 11 23:31:21.657707 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657704 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.658028 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.657839 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 11 23:31:21.659264 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.659246 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 11 23:31:21.659516 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.659501 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 11 23:31:21.660136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.660118 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.662356 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.662340 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-vhqzf\"" Jul 11 23:31:21.662596 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.662581 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 11 23:31:21.662653 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.662610 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 11 23:31:21.663381 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.663363 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.665778 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.665757 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tc9wg\"" Jul 11 23:31:21.665778 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.665771 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.665778 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.665764 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 11 23:31:21.665976 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.665772 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.666036 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.665981 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.668643 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.668624 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.669167 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.669148 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 11 23:31:21.669263 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.669180 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-zs5np\"" Jul 11 23:31:21.669439 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.669424 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.669507 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.669460 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.671044 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.671025 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 11 23:31:21.671044 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.671042 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 11 23:31:21.671193 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.671028 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-gt2f5\"" Jul 11 23:31:21.671692 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.671674 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.673838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.673820 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.674306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674289 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-ddvdx\"" Jul 11 23:31:21.674395 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674342 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.674570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674544 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal"] Jul 11 23:31:21.674570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674579 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal"] Jul 11 23:31:21.674709 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.674824 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.674807 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 11 23:31:21.677017 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.676988 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 11 23:31:21.677017 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.677006 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wxdp6\"" Jul 11 23:31:21.677427 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.677411 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 11 23:31:21.706670 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.706651 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-mct44" Jul 11 23:31:21.713066 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.713047 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-mct44" Jul 11 23:31:21.742266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.742247 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 11 23:31:21.747144 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747114 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-kubelet\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747268 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747156 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-socket-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.747268 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-etc-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.747268 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-kubernetes\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747268 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747235 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-var-lib-kubelet\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747268 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747262 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-btblt\" (UniqueName: \"kubernetes.io/projected/338bdff5-9113-462b-a1db-947024a1db1d-kube-api-access-btblt\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747289 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-netns\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747314 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-node-log\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747338 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-netd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b085832-ada8-407e-8fb8-ab9a7c310ee9-hosts-file\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747419 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysconfig\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747448 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7b085832-ada8-407e-8fb8-ab9a7c310ee9-hosts-file\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747491 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8nr7\" (UniqueName: \"kubernetes.io/projected/f534644e-c7d0-4666-b122-917dbcd06661-kube-api-access-r8nr7\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b754z\" (UniqueName: \"kubernetes.io/projected/fd67cf09-fc47-41a1-bb87-13f015fab302-kube-api-access-b754z\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747558 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-host\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747595 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-conf\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cni-binary-copy\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747661 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-multus-certs\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747679 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-dbus\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747696 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-systemd\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-tmp\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747752 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747777 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-conf-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-device-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747844 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tcbts\" (UniqueName: \"kubernetes.io/projected/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-kube-api-access-tcbts\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747869 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747891 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-run\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747912 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-sys\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747928 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-system-cni-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747944 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.747955 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.747960 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-modprobe-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-systemd-units\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748037 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-ovn\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748067 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-script-lib\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748092 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-lib-modules\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748116 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-hostroot\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748137 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-agent-certs\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-konnectivity-ca\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748182 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-serviceca\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtsqm\" (UniqueName: \"kubernetes.io/projected/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-kube-api-access-dtsqm\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748240 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-host\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748265 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748295 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25fk2\" (UniqueName: \"kubernetes.io/projected/5edd8b3d-0380-41e5-b734-655dd000e945-kube-api-access-25fk2\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748331 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748375 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zml4b\" (UniqueName: \"kubernetes.io/projected/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-kube-api-access-zml4b\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.748696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748432 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cnibin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-bin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748482 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-daemon-config\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748530 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-etc-kubernetes\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748569 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-iptables-alerter-script\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-etc-tuned\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748630 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-netns\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748655 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-registration-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748682 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-kubelet\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-system-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748855 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-slash\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748880 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-systemd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748921 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-bin\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748962 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-env-overrides\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.749499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.748991 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5edd8b3d-0380-41e5-b734-655dd000e945-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749031 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-kubelet-config\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749058 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-sys-fs\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749093 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-var-lib-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749145 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-config\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749191 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749215 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-os-release\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749257 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-k8s-cni-cncf-io\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749288 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749304 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-log-socket\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749344 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b085832-ada8-407e-8fb8-ab9a7c310ee9-tmp-dir\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749372 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cnibin\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749397 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-os-release\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4pxqd\" (UniqueName: \"kubernetes.io/projected/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-kube-api-access-4pxqd\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.750284 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-socket-dir-parent\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.751047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749489 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.751047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2fllx\" (UniqueName: \"kubernetes.io/projected/7b085832-ada8-407e-8fb8-ab9a7c310ee9-kube-api-access-2fllx\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.751047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749532 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-host-slash\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.751047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-multus\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.751047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.749688 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7b085832-ada8-407e-8fb8-ab9a7c310ee9-tmp-dir\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.755082 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.755060 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 11 23:31:21.758161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.758123 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2fllx\" (UniqueName: \"kubernetes.io/projected/7b085832-ada8-407e-8fb8-ab9a7c310ee9-kube-api-access-2fllx\") pod \"node-resolver-2qvz4\" (UID: \"7b085832-ada8-407e-8fb8-ab9a7c310ee9\") " pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.829127 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:21.829096 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod16a2c0a00cdbe9d77d0bff3f91f1c3e7.slice/crio-715282a2f1c6924d90fa30f7b19bf598cc732e391053dca24a35bbbd34b86185 WatchSource:0}: Error finding container 715282a2f1c6924d90fa30f7b19bf598cc732e391053dca24a35bbbd34b86185: Status 404 returned error can't find the container with id 715282a2f1c6924d90fa30f7b19bf598cc732e391053dca24a35bbbd34b86185 Jul 11 23:31:21.835482 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.835464 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 23:31:21.836007 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:21.835984 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod281d1b3a45b212ba9681d002edffe820.slice/crio-21123d68f50a4b8bfa122b13cf01ae563c41e5440da0d2bf6ba65ace5ec5ebae WatchSource:0}: Error finding container 21123d68f50a4b8bfa122b13cf01ae563c41e5440da0d2bf6ba65ace5ec5ebae: Status 404 returned error can't find the container with id 21123d68f50a4b8bfa122b13cf01ae563c41e5440da0d2bf6ba65ace5ec5ebae Jul 11 23:31:21.849803 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849781 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-agent-certs\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849809 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-konnectivity-ca\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-serviceca\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtsqm\" (UniqueName: \"kubernetes.io/projected/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-kube-api-access-dtsqm\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-host\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.849917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849913 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-25fk2\" (UniqueName: \"kubernetes.io/projected/5edd8b3d-0380-41e5-b734-655dd000e945-kube-api-access-25fk2\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849940 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849966 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849971 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-host\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.849992 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zml4b\" (UniqueName: \"kubernetes.io/projected/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-kube-api-access-zml4b\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850017 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cnibin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-bin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850062 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850328 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-bin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850365 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-konnectivity-ca\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850371 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-serviceca\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850540 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850551 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-daemon-config\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-etc-kubernetes\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-iptables-alerter-script\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850626 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cnibin\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-etc-tuned\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-netns\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850679 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-etc-kubernetes\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-registration-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850753 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-netns\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-kubelet\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850808 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-system-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850846 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-etc-selinux\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850859 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-slash\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850884 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-systemd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-kubelet\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.850914 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850909 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-bin\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-env-overrides\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850944 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-system-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850947 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-registration-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850960 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5edd8b3d-0380-41e5-b734-655dd000e945-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.850998 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-kubelet-config\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851019 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-bin\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851026 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-sys-fs\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851052 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-var-lib-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851073 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-systemd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-config\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851126 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851141 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-daemon-config\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851159 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-os-release\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-k8s-cni-cncf-io\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851200 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-os-release\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.851850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851235 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-slash\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851248 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-k8s-cni-cncf-io\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851294 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-log-socket\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851300 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-kubelet-config\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851316 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-sys-fs\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851327 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851357 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cnibin\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851365 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-var-lib-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851386 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-os-release\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851396 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cni-binary-copy\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4pxqd\" (UniqueName: \"kubernetes.io/projected/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-kube-api-access-4pxqd\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851442 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-log-socket\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-socket-dir-parent\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-kubelet-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851499 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-socket-dir-parent\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.852864 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851497 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-cnibin\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851513 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-host-slash\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851412 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-multus\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851608 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-os-release\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851622 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-host-slash\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851657 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-kubelet\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851667 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-cni-multus\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851711 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-var-lib-kubelet\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.851721 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851743 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-socket-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-etc-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-kubernetes\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.851819 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:22.351795406 +0000 UTC m=+2.103542890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851851 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-var-lib-kubelet\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851893 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-btblt\" (UniqueName: \"kubernetes.io/projected/338bdff5-9113-462b-a1db-947024a1db1d-kube-api-access-btblt\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851897 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-kubernetes\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.853713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851921 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-netns\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851921 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-iptables-alerter-script\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.851993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-run-netns\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852032 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-etc-openvswitch\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-node-log\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-netd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852141 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysconfig\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852143 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-config\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852170 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-env-overrides\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852172 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852274 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-var-lib-kubelet\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852181 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-node-log\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852212 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-host-cni-netd\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852244 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysconfig\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8nr7\" (UniqueName: \"kubernetes.io/projected/f534644e-c7d0-4666-b122-917dbcd06661-kube-api-access-r8nr7\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-socket-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b754z\" (UniqueName: \"kubernetes.io/projected/fd67cf09-fc47-41a1-bb87-13f015fab302-kube-api-access-b754z\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.854332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-host\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852390 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-conf\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852415 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cni-binary-copy\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-multus-certs\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852445 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852462 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-host\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-dbus\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852521 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-systemd\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-tmp\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852573 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852601 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-sysctl-conf\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-conf-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-systemd\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852646 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-device-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tcbts\" (UniqueName: \"kubernetes.io/projected/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-kube-api-access-tcbts\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-cni-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-run\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852741 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-host-run-multus-certs\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-sys\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852781 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-multus-conf-dir\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852789 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-system-cni-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852819 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852831 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-run\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852834 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/fd67cf09-fc47-41a1-bb87-13f015fab302-device-dir\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852848 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-modprobe-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852875 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-systemd-units\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852899 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-sys\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.852909 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852933 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-cni-binary-copy\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852937 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-ovn\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.852968 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:22.35295169 +0000 UTC m=+2.104699173 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852900 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-run-ovn\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/5edd8b3d-0380-41e5-b734-655dd000e945-systemd-units\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853004 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-script-lib\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.855639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852964 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-system-cni-dir\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853001 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-etc-modprobe-d\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853040 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-lib-modules\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853064 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-hostroot\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853127 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-hostroot\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853220 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f534644e-c7d0-4666-b122-917dbcd06661-lib-modules\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.852710 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/f5de7bc3-7a27-4033-9303-380b33e254f9-dbus\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853592 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/5edd8b3d-0380-41e5-b734-655dd000e945-ovnkube-script-lib\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853615 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-etc-tuned\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.853998 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.854929 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/f534644e-c7d0-4666-b122-917dbcd06661-tmp\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.854951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab-agent-certs\") pod \"konnectivity-agent-rw7vw\" (UID: \"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab\") " pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:21.856217 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.854996 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/5edd8b3d-0380-41e5-b734-655dd000e945-ovn-node-metrics-cert\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.866419 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.866400 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:21.866419 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.866421 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:21.866586 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.866434 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:21.866586 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:21.866511 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:22.36647381 +0000 UTC m=+2.118221309 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:21.868779 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.868750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tcbts\" (UniqueName: \"kubernetes.io/projected/feca8d66-b572-4fca-bcc0-2eda7e4a1d2b-kube-api-access-tcbts\") pod \"multus-88pmr\" (UID: \"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b\") " pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.869078 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.869054 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-btblt\" (UniqueName: \"kubernetes.io/projected/338bdff5-9113-462b-a1db-947024a1db1d-kube-api-access-btblt\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:21.869382 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.869350 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtsqm\" (UniqueName: \"kubernetes.io/projected/c3208457-f67d-4e7d-bfe6-b57b1baa6d7b-kube-api-access-dtsqm\") pod \"node-ca-m92gl\" (UID: \"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b\") " pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:21.869646 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.869628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-25fk2\" (UniqueName: \"kubernetes.io/projected/5edd8b3d-0380-41e5-b734-655dd000e945-kube-api-access-25fk2\") pod \"ovnkube-node-dwvq7\" (UID: \"5edd8b3d-0380-41e5-b734-655dd000e945\") " pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:21.869776 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.869756 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8nr7\" (UniqueName: \"kubernetes.io/projected/f534644e-c7d0-4666-b122-917dbcd06661-kube-api-access-r8nr7\") pod \"tuned-7k495\" (UID: \"f534644e-c7d0-4666-b122-917dbcd06661\") " pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:21.869826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.869798 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zml4b\" (UniqueName: \"kubernetes.io/projected/77b79610-2e35-4d57-bca6-5ac0ee3d0b2f-kube-api-access-zml4b\") pod \"multus-additional-cni-plugins-mbh6w\" (UID: \"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f\") " pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:21.870203 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.870184 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4pxqd\" (UniqueName: \"kubernetes.io/projected/d73b9b83-b8d2-485a-9238-ddf2f1aa58d7-kube-api-access-4pxqd\") pod \"iptables-alerter-8p8bs\" (UID: \"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7\") " pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:21.870240 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.870187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b754z\" (UniqueName: \"kubernetes.io/projected/fd67cf09-fc47-41a1-bb87-13f015fab302-kube-api-access-b754z\") pod \"aws-ebs-csi-driver-node-vfpxc\" (UID: \"fd67cf09-fc47-41a1-bb87-13f015fab302\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:21.972429 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.972399 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-2qvz4" Jul 11 23:31:21.982569 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:21.982544 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b085832_ada8_407e_8fb8_ab9a7c310ee9.slice/crio-aeac5d10897655e3531f91fae6556a887db7cdd6f8c7d5150e23d43dc0949ee5 WatchSource:0}: Error finding container aeac5d10897655e3531f91fae6556a887db7cdd6f8c7d5150e23d43dc0949ee5: Status 404 returned error can't find the container with id aeac5d10897655e3531f91fae6556a887db7cdd6f8c7d5150e23d43dc0949ee5 Jul 11 23:31:21.984414 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:21.984398 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-88pmr" Jul 11 23:31:21.993318 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:21.993297 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfeca8d66_b572_4fca_bcc0_2eda7e4a1d2b.slice/crio-f94eddba6044604ea839c87aec288b582d592cee3579b9f15d1bc002811b4468 WatchSource:0}: Error finding container f94eddba6044604ea839c87aec288b582d592cee3579b9f15d1bc002811b4468: Status 404 returned error can't find the container with id f94eddba6044604ea839c87aec288b582d592cee3579b9f15d1bc002811b4468 Jul 11 23:31:22.003688 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.003670 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:22.013119 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.013095 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5edd8b3d_0380_41e5_b734_655dd000e945.slice/crio-4002eebec470fb51cf42ef10406dd9f277237000378f85ac42dc577194376c34 WatchSource:0}: Error finding container 4002eebec470fb51cf42ef10406dd9f277237000378f85ac42dc577194376c34: Status 404 returned error can't find the container with id 4002eebec470fb51cf42ef10406dd9f277237000378f85ac42dc577194376c34 Jul 11 23:31:22.020208 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.020192 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:22.026191 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.026169 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" Jul 11 23:31:22.028093 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.028070 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod886f3bd4_de43_4d6c_a07c_0b3b3c76a0ab.slice/crio-fa3540e509b42401b4746d117cbcafaa7eef39694e8b6fb8b5e8e9b7d422b7b6 WatchSource:0}: Error finding container fa3540e509b42401b4746d117cbcafaa7eef39694e8b6fb8b5e8e9b7d422b7b6: Status 404 returned error can't find the container with id fa3540e509b42401b4746d117cbcafaa7eef39694e8b6fb8b5e8e9b7d422b7b6 Jul 11 23:31:22.034659 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.034635 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd67cf09_fc47_41a1_bb87_13f015fab302.slice/crio-6304a80b11ae11b1e6ac0d164ab8b7d509961bb8038f1837bb1d673c396dc555 WatchSource:0}: Error finding container 6304a80b11ae11b1e6ac0d164ab8b7d509961bb8038f1837bb1d673c396dc555: Status 404 returned error can't find the container with id 6304a80b11ae11b1e6ac0d164ab8b7d509961bb8038f1837bb1d673c396dc555 Jul 11 23:31:22.047117 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.047091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-m92gl" Jul 11 23:31:22.055722 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.055679 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3208457_f67d_4e7d_bfe6_b57b1baa6d7b.slice/crio-45c85dd564c33c6e2f3b39ea27c4394a2710ba6e339425cf76a01fab6e6480bf WatchSource:0}: Error finding container 45c85dd564c33c6e2f3b39ea27c4394a2710ba6e339425cf76a01fab6e6480bf: Status 404 returned error can't find the container with id 45c85dd564c33c6e2f3b39ea27c4394a2710ba6e339425cf76a01fab6e6480bf Jul 11 23:31:22.060820 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.060801 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:22.061913 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.061898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" Jul 11 23:31:22.070215 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.070195 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77b79610_2e35_4d57_bca6_5ac0ee3d0b2f.slice/crio-9ac6e03fb17647092af829fc8edecab44c636cf395e31920ca6d16c59509f2b8 WatchSource:0}: Error finding container 9ac6e03fb17647092af829fc8edecab44c636cf395e31920ca6d16c59509f2b8: Status 404 returned error can't find the container with id 9ac6e03fb17647092af829fc8edecab44c636cf395e31920ca6d16c59509f2b8 Jul 11 23:31:22.079016 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.078999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-8p8bs" Jul 11 23:31:22.084018 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.083999 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-7k495" Jul 11 23:31:22.088004 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.087983 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd73b9b83_b8d2_485a_9238_ddf2f1aa58d7.slice/crio-c52ef2def56109b533bbd5d48cacfdf3afc2f244c6e8ef4a826580f28fddc408 WatchSource:0}: Error finding container c52ef2def56109b533bbd5d48cacfdf3afc2f244c6e8ef4a826580f28fddc408: Status 404 returned error can't find the container with id c52ef2def56109b533bbd5d48cacfdf3afc2f244c6e8ef4a826580f28fddc408 Jul 11 23:31:22.092326 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:31:22.092303 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf534644e_c7d0_4666_b122_917dbcd06661.slice/crio-9c10e3445650cf5a5fecd947b29896f87281e176f435c216470d2a07e5bb6f9d WatchSource:0}: Error finding container 9c10e3445650cf5a5fecd947b29896f87281e176f435c216470d2a07e5bb6f9d: Status 404 returned error can't find the container with id 9c10e3445650cf5a5fecd947b29896f87281e176f435c216470d2a07e5bb6f9d Jul 11 23:31:22.243434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.243398 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:22.357917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.357298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:22.357917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.357384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:22.357917 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.357538 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:22.357917 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.357602 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:23.357579662 +0000 UTC m=+3.109327160 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:22.358259 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.357932 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:22.358259 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.358009 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:23.357989214 +0000 UTC m=+3.109736704 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:22.457966 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.457929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:22.458163 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.458146 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:22.458237 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.458174 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:22.458237 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.458188 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:22.458351 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:22.458246 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:23.45822485 +0000 UTC m=+3.209972335 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:22.716410 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.716306 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 23:26:22 +0000 UTC" deadline="2028-02-27 09:11:09.356630497 +0000 UTC" Jul 11 23:31:22.716410 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.716358 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14289h39m46.640278224s" Jul 11 23:31:22.741007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.740976 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:22.803831 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.803700 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerStarted","Data":"9ac6e03fb17647092af829fc8edecab44c636cf395e31920ca6d16c59509f2b8"} Jul 11 23:31:22.834094 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.834053 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m92gl" event={"ID":"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b","Type":"ContainerStarted","Data":"45c85dd564c33c6e2f3b39ea27c4394a2710ba6e339425cf76a01fab6e6480bf"} Jul 11 23:31:22.856704 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.856648 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rw7vw" event={"ID":"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab","Type":"ContainerStarted","Data":"fa3540e509b42401b4746d117cbcafaa7eef39694e8b6fb8b5e8e9b7d422b7b6"} Jul 11 23:31:22.886246 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.886177 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-88pmr" event={"ID":"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b","Type":"ContainerStarted","Data":"f94eddba6044604ea839c87aec288b582d592cee3579b9f15d1bc002811b4468"} Jul 11 23:31:22.907967 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.907874 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2qvz4" event={"ID":"7b085832-ada8-407e-8fb8-ab9a7c310ee9","Type":"ContainerStarted","Data":"aeac5d10897655e3531f91fae6556a887db7cdd6f8c7d5150e23d43dc0949ee5"} Jul 11 23:31:22.926246 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.925716 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7k495" event={"ID":"f534644e-c7d0-4666-b122-917dbcd06661","Type":"ContainerStarted","Data":"9c10e3445650cf5a5fecd947b29896f87281e176f435c216470d2a07e5bb6f9d"} Jul 11 23:31:22.931858 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.931616 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8p8bs" event={"ID":"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7","Type":"ContainerStarted","Data":"c52ef2def56109b533bbd5d48cacfdf3afc2f244c6e8ef4a826580f28fddc408"} Jul 11 23:31:22.948570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.946722 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" event={"ID":"fd67cf09-fc47-41a1-bb87-13f015fab302","Type":"ContainerStarted","Data":"6304a80b11ae11b1e6ac0d164ab8b7d509961bb8038f1837bb1d673c396dc555"} Jul 11 23:31:22.974506 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.974410 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"4002eebec470fb51cf42ef10406dd9f277237000378f85ac42dc577194376c34"} Jul 11 23:31:22.984610 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:22.984580 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerStarted","Data":"21123d68f50a4b8bfa122b13cf01ae563c41e5440da0d2bf6ba65ace5ec5ebae"} Jul 11 23:31:23.001638 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.001512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" event={"ID":"16a2c0a00cdbe9d77d0bff3f91f1c3e7","Type":"ContainerStarted","Data":"715282a2f1c6924d90fa30f7b19bf598cc732e391053dca24a35bbbd34b86185"} Jul 11 23:31:23.365361 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.365257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:23.365361 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.365354 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:23.365604 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.365497 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:23.365604 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.365582 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:25.365560819 +0000 UTC m=+5.117308306 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:23.366040 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.366018 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:23.366136 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.366076 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:25.366059832 +0000 UTC m=+5.117807331 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:23.466578 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.466459 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:23.466837 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.466641 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:23.466837 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.466661 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:23.466837 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.466677 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:23.466837 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.466744 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:25.466724605 +0000 UTC m=+5.218472090 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:23.717573 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.717427 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-10 23:26:22 +0000 UTC" deadline="2028-01-11 07:14:38.376236644 +0000 UTC" Jul 11 23:31:23.717573 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.717519 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13159h43m14.658731911s" Jul 11 23:31:23.782872 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.782842 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:23.783045 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.782973 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:23.783402 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.783381 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:23.783513 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.783494 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:23.783601 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:23.783579 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:23.783677 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:23.783656 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:24.063189 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:24.063036 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 11 23:31:25.384900 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.384863 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:25.385389 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.384963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:25.385389 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.385096 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:25.385389 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.385155 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:29.385137271 +0000 UTC m=+9.136884772 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:25.385676 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.385568 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:25.385676 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.385622 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:29.385602798 +0000 UTC m=+9.137350321 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:25.485944 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.485835 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:25.486125 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.486046 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:25.486125 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.486074 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:25.486125 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.486087 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:25.486294 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.486146 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:29.486126005 +0000 UTC m=+9.237873490 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:25.782586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.782472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:25.782756 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.782485 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:25.782756 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.782630 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:25.782756 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.782712 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:25.782756 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:25.782472 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:25.782968 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:25.782824 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:27.782464 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:27.782425 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:27.782994 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:27.782586 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:27.782994 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:27.782589 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:27.782994 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:27.782721 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:27.782994 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:27.782763 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:27.782994 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:27.782878 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:29.421134 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.420383 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:29.421134 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.420480 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:29.421134 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.420642 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:29.421134 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.420708 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:37.420687531 +0000 UTC m=+17.172435018 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:29.421134 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.421137 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:29.421902 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.421185 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:37.421168309 +0000 UTC m=+17.172915797 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:29.521419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.521348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:29.521629 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.521533 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:29.521629 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.521572 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:29.521629 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.521590 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:29.521809 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.521661 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:37.521640184 +0000 UTC m=+17.273387691 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:29.782838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.782753 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:29.783017 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.782897 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:29.783462 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.783313 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:29.783462 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.783438 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:29.783462 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:29.783468 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:29.783712 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:29.783538 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:31.782228 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:31.782182 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:31.782721 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:31.782320 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:31.782721 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:31.782332 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:31.782721 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:31.782455 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:31.782721 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:31.782534 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:31.782721 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:31.782605 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:33.783165 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:33.782945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:33.783612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:33.782945 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:33.783612 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:33.783275 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:33.783612 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:33.783351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:33.783612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:33.782949 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:33.783612 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:33.783439 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:35.782461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:35.782427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:35.782461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:35.782443 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:35.782938 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:35.782567 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:35.782938 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:35.782634 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:35.782938 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:35.782682 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:35.782938 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:35.782753 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:37.476523 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.476482 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:37.476970 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.476569 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:37.476970 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.476642 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:37.476970 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.476674 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:37.476970 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.476711 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:53.476690707 +0000 UTC m=+33.228438189 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:37.476970 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.476727 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:31:53.476720582 +0000 UTC m=+33.228468062 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:37.577156 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.577121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:37.577353 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.577293 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:37.577353 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.577314 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:37.577353 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.577324 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:37.577524 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.577382 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:31:53.577363967 +0000 UTC m=+33.329111468 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:37.782612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.782529 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:37.782772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.782534 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:37.782772 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.782649 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:37.782772 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.782736 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:37.782772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:37.782540 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:37.782960 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:37.782821 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:39.782416 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:39.782383 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:39.782416 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:39.782426 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:39.782926 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:39.782536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:39.782926 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:39.782550 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:39.782926 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:39.782650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:39.782926 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:39.782729 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:41.782144 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:41.782101 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:41.782608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:41.782208 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:41.782608 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:41.782210 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:41.782608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:41.782218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:41.782608 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:41.782316 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:41.782608 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:41.782413 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:43.065718 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.065362 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-7k495" event={"ID":"f534644e-c7d0-4666-b122-917dbcd06661","Type":"ContainerStarted","Data":"801ed965d19dc042ff105fb7f5e9d7b5f62fb174355bb94adcefbe9c70292bc8"} Jul 11 23:31:43.066738 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.066713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" event={"ID":"fd67cf09-fc47-41a1-bb87-13f015fab302","Type":"ContainerStarted","Data":"45497d5432c6f040bae9f2b8f37b8cd87b69d2db46ea91347ff210c00a477167"} Jul 11 23:31:43.068617 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.068598 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:31:43.068931 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.068909 2580 generic.go:356] "Generic (PLEG): container finished" podID="5edd8b3d-0380-41e5-b734-655dd000e945" containerID="1b310535037049c86d558de01ca0f0f3608df8599de65c8fe416f79632668eab" exitCode=1 Jul 11 23:31:43.069013 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.068969 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"4706e11871fb02aea629cac8bd5c4f6e9ba50cdb277573a365047e101664e721"} Jul 11 23:31:43.069066 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.069021 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"2e431d17e1cfde80cba6cc8690aa3291ba6dcefa3a11e199ed3eb2307964966f"} Jul 11 23:31:43.069066 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.069051 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerDied","Data":"1b310535037049c86d558de01ca0f0f3608df8599de65c8fe416f79632668eab"} Jul 11 23:31:43.069149 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.069082 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"58c9a47570d5a070ab04ad3cdf8bba506a8d6c1cc8d8b57949bced19715dde74"} Jul 11 23:31:43.070222 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.070205 2580 generic.go:356] "Generic (PLEG): container finished" podID="281d1b3a45b212ba9681d002edffe820" containerID="eb1a200c64668e5d6a86dd7709d6bd4ac04d66c1e67b530f50c49fbc5c82b215" exitCode=0 Jul 11 23:31:43.070293 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.070271 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerDied","Data":"eb1a200c64668e5d6a86dd7709d6bd4ac04d66c1e67b530f50c49fbc5c82b215"} Jul 11 23:31:43.071513 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.071494 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" event={"ID":"16a2c0a00cdbe9d77d0bff3f91f1c3e7","Type":"ContainerStarted","Data":"54ec5484c1cb0be32a9c5ecc869da1a7574c793c876bc928a59af60d19dd9733"} Jul 11 23:31:43.072745 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.072728 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="2cd4d98d3836f8ba271afd86a06105798a844f352c11d5e9b0c00bfe4513f26a" exitCode=0 Jul 11 23:31:43.072819 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.072783 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"2cd4d98d3836f8ba271afd86a06105798a844f352c11d5e9b0c00bfe4513f26a"} Jul 11 23:31:43.074226 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.074206 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-m92gl" event={"ID":"c3208457-f67d-4e7d-bfe6-b57b1baa6d7b","Type":"ContainerStarted","Data":"df2dd9ffc1db9ab2cc74db99a1b1a46a2fa08b37332ad0c9767354d04bab8fa2"} Jul 11 23:31:43.075618 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.075592 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-rw7vw" event={"ID":"886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab","Type":"ContainerStarted","Data":"b2438865b68bde2757218c36cca184b10fe1b965f69107bc787168d04246dd2d"} Jul 11 23:31:43.076865 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.076842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-88pmr" event={"ID":"feca8d66-b572-4fca-bcc0-2eda7e4a1d2b","Type":"ContainerStarted","Data":"785666aa411c95c0fc543344acac149ed29f0c6baace79e4bcbf96a83b72cc86"} Jul 11 23:31:43.077952 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.077934 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-2qvz4" event={"ID":"7b085832-ada8-407e-8fb8-ab9a7c310ee9","Type":"ContainerStarted","Data":"7f5db391042b540f90f70df933215e7db8f735b7e111d5271b3559369353f63d"} Jul 11 23:31:43.083348 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.083311 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-7k495" podStartSLOduration=1.669077513 podStartE2EDuration="22.083300071s" totalImagesPullingTime="20.414222558s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.082759591 +0000 UTC m=+22.834507095" watchObservedRunningTime="2026-07-11 23:31:43.083300071 +0000 UTC m=+22.835047573" Jul 11 23:31:43.151968 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.151927 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-m92gl" podStartSLOduration=1.6979087019999999 podStartE2EDuration="22.151912664s" totalImagesPullingTime="20.454003962s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.151878751 +0000 UTC m=+22.903626275" watchObservedRunningTime="2026-07-11 23:31:43.151912664 +0000 UTC m=+22.903660168" Jul 11 23:31:43.152140 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.152116 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-140-223.ec2.internal" podStartSLOduration=21.152110441 podStartE2EDuration="21.152110441s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.137232108 +0000 UTC m=+22.888979611" watchObservedRunningTime="2026-07-11 23:31:43.152110441 +0000 UTC m=+22.903857944" Jul 11 23:31:43.165278 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.165227 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-rw7vw" podStartSLOduration=1.683531383 podStartE2EDuration="22.165207606s" totalImagesPullingTime="20.481676223s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.165195314 +0000 UTC m=+22.916942816" watchObservedRunningTime="2026-07-11 23:31:43.165207606 +0000 UTC m=+22.916955092" Jul 11 23:31:43.177589 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.177461 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-2qvz4" podStartSLOduration=1.651695213 podStartE2EDuration="22.177444488s" totalImagesPullingTime="20.525749275s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.17729308 +0000 UTC m=+22.929040573" watchObservedRunningTime="2026-07-11 23:31:43.177444488 +0000 UTC m=+22.929191991" Jul 11 23:31:43.193253 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.193210 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-88pmr" podStartSLOduration=1.626002394 podStartE2EDuration="22.193195559s" totalImagesPullingTime="20.567193165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:43.193079333 +0000 UTC m=+22.944826847" watchObservedRunningTime="2026-07-11 23:31:43.193195559 +0000 UTC m=+22.944943062" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.782284 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.782404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:43.782409 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:43.782512 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:43.782561 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:43.782697 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:43.782623 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:44.084212 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.084182 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:31:44.084696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.084591 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"93d65f82ec354d8eb3b17db9765736f121d6cfdc76c59a2272b34171cbd65ea9"} Jul 11 23:31:44.084696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.084659 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"654c3aa23b8f61cc0b7e0b72d507225439d212ca44d67fc889ff7cba9dff0f43"} Jul 11 23:31:44.086558 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.086505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" event={"ID":"281d1b3a45b212ba9681d002edffe820","Type":"ContainerStarted","Data":"f00dd534924adda9606d144a6e35d7884ebef70d62a2d6f5a08a218425dd67d9"} Jul 11 23:31:44.088162 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.088128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-8p8bs" event={"ID":"d73b9b83-b8d2-485a-9238-ddf2f1aa58d7","Type":"ContainerStarted","Data":"a1852c45d83f829b60b8ebf279c704640f5db56ac4ae36ac846886d9c62f191e"} Jul 11 23:31:44.100272 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.100185 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-140-223.ec2.internal" podStartSLOduration=22.100169721 podStartE2EDuration="22.100169721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:22 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:44.099574032 +0000 UTC m=+23.851321535" watchObservedRunningTime="2026-07-11 23:31:44.100169721 +0000 UTC m=+23.851917223" Jul 11 23:31:44.113179 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.113136 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-8p8bs" podStartSLOduration=2.684406734 podStartE2EDuration="23.113120529s" totalImagesPullingTime="20.428713795s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:44.112213881 +0000 UTC m=+23.863961408" watchObservedRunningTime="2026-07-11 23:31:44.113120529 +0000 UTC m=+23.864868122" Jul 11 23:31:44.308060 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.307938 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 11 23:31:44.708088 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.707969 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-11T23:31:44.307959974Z","UUID":"4c653cfd-253a-4aa2-b093-5f0bbaf48562","Handler":null,"Name":"","Endpoint":""} Jul 11 23:31:44.709693 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.709665 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 11 23:31:44.709836 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:44.709706 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 11 23:31:45.092698 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:45.092662 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" event={"ID":"fd67cf09-fc47-41a1-bb87-13f015fab302","Type":"ContainerStarted","Data":"d21f9a90b390bee7e232942cee107ac22936032eca8d229b0e0d95f761881569"} Jul 11 23:31:45.782464 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:45.782283 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:45.782611 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:45.782309 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:45.782611 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:45.782565 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:45.782681 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:45.782612 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:45.782681 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:45.782283 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:45.782745 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:45.782702 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:46.098224 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:46.098201 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:31:46.098696 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:46.098666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"d79ee477fb6912b87e593135bdc2c04c136a5dd4cb070f812e29a3c1abbadd1d"} Jul 11 23:31:46.101004 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:46.100971 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" event={"ID":"fd67cf09-fc47-41a1-bb87-13f015fab302","Type":"ContainerStarted","Data":"c8ae6e07d9b0f56ac8040f724771f572b7dcef69257d3b580bfda9ab4ea38f4c"} Jul 11 23:31:46.117996 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:46.117948 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-vfpxc" podStartSLOduration=2.024578551 podStartE2EDuration="25.117935197s" totalImagesPullingTime="23.093356646s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:46.117848146 +0000 UTC m=+25.869595669" watchObservedRunningTime="2026-07-11 23:31:46.117935197 +0000 UTC m=+25.869682699" Jul 11 23:31:47.782757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:47.782522 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:47.782757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:47.782533 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:47.783406 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:47.782769 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:47.783406 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:47.782538 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:47.783406 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:47.782829 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:47.783406 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:47.782910 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:47.967623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:47.967594 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:47.968127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:47.968109 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:48.106735 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.106649 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="20138c62a16d3b90801a62ab910758cd73999b5dafe4d81d05a9a8966a3a0067" exitCode=0 Jul 11 23:31:48.106735 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.106713 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"20138c62a16d3b90801a62ab910758cd73999b5dafe4d81d05a9a8966a3a0067"} Jul 11 23:31:48.110039 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.110020 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:31:48.110408 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.110379 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"066ac494ef7dfd242a2fce756314c80792a0e9f98780df15fb9b9f941b161672"} Jul 11 23:31:48.110704 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.110685 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:48.110816 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.110757 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:48.110855 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.110814 2580 scope.go:117] "RemoveContainer" containerID="1b310535037049c86d558de01ca0f0f3608df8599de65c8fe416f79632668eab" Jul 11 23:31:48.126612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:48.126594 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:49.116079 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.116054 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:31:49.116564 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.116380 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" event={"ID":"5edd8b3d-0380-41e5-b734-655dd000e945","Type":"ContainerStarted","Data":"7fd70b7fb8280a5769ffbd952d54268719156ea237b0486e9202f15b8d15417e"} Jul 11 23:31:49.116771 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.116750 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:49.135860 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.135835 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:31:49.143763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.143717 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" podStartSLOduration=7.561997055 podStartE2EDuration="28.143702736s" totalImagesPullingTime="20.581705681s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:49.142306659 +0000 UTC m=+28.894054187" watchObservedRunningTime="2026-07-11 23:31:49.143702736 +0000 UTC m=+28.895450239" Jul 11 23:31:49.495390 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.495365 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gstql"] Jul 11 23:31:49.495675 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.495536 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:49.495675 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:49.495650 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:49.498522 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.498500 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cdt7s"] Jul 11 23:31:49.498631 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.498592 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:49.498708 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:49.498682 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:49.499177 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.499155 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b465v"] Jul 11 23:31:49.499259 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.499246 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:49.499349 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:49.499329 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:49.652592 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.652552 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:49.652759 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.652695 2580 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 11 23:31:49.653178 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:49.653158 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-rw7vw" Jul 11 23:31:50.120410 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:50.120379 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="55c8cb9976633e4013ac44ed1407b248c4a2e76b58c13495ccb5223994014dda" exitCode=0 Jul 11 23:31:50.120803 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:50.120464 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"55c8cb9976633e4013ac44ed1407b248c4a2e76b58c13495ccb5223994014dda"} Jul 11 23:31:50.783829 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:50.783625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:50.783986 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:50.783711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:50.783986 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:50.783901 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:50.784082 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:50.784003 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:51.782242 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:51.782207 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:51.782739 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:51.782309 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:52.128117 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:52.127940 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="33da61e26da6294d3c2362a6ee3fe99a6909aec2e17619f5b07dfbe4b1669ae7" exitCode=0 Jul 11 23:31:52.128117 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:52.128013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"33da61e26da6294d3c2362a6ee3fe99a6909aec2e17619f5b07dfbe4b1669ae7"} Jul 11 23:31:52.486398 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:52.486373 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2qvz4_7b085832-ada8-407e-8fb8-ab9a7c310ee9/dns-node-resolver/0.log" Jul 11 23:31:52.782524 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:52.782427 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:52.782524 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:52.782451 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:52.783282 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:52.782598 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:52.783282 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:52.782721 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:53.494005 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:53.493973 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:53.494182 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:53.494058 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:53.494182 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.494154 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:53.494182 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.494170 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:53.494312 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.494221 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret podName:f5de7bc3-7a27-4033-9303-380b33e254f9 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:25.494205459 +0000 UTC m=+65.245952953 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret") pod "global-pull-secret-syncer-gstql" (UID: "f5de7bc3-7a27-4033-9303-380b33e254f9") : object "kube-system"/"original-pull-secret" not registered Jul 11 23:31:53.494312 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.494237 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs podName:338bdff5-9113-462b-a1db-947024a1db1d nodeName:}" failed. No retries permitted until 2026-07-11 23:32:25.494228592 +0000 UTC m=+65.245976097 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs") pod "network-metrics-daemon-cdt7s" (UID: "338bdff5-9113-462b-a1db-947024a1db1d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 11 23:31:53.594650 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:53.594615 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:53.594903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.594781 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 11 23:31:53.594903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.594806 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 11 23:31:53.594903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.594818 2580 projected.go:196] Error preparing data for projected volume kube-api-access-m9l67 for pod openshift-network-diagnostics/network-check-target-b465v: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:53.594903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.594876 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67 podName:0d21d616-2e36-419b-945e-03b79b94946d nodeName:}" failed. No retries permitted until 2026-07-11 23:32:25.594859224 +0000 UTC m=+65.346606711 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-m9l67" (UniqueName: "kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67") pod "network-check-target-b465v" (UID: "0d21d616-2e36-419b-945e-03b79b94946d") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 11 23:31:53.668447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:53.668414 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m92gl_c3208457-f67d-4e7d-bfe6-b57b1baa6d7b/node-ca/0.log" Jul 11 23:31:53.782342 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:53.782269 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:53.782505 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:53.782369 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:54.782553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:54.782525 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:54.783167 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:54.782574 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:54.783167 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:54.782652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:54.783167 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:54.782801 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:55.782806 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:55.782770 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:55.783259 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:55.782882 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:56.782890 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:56.782804 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:56.783246 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:56.782921 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:56.783246 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:56.782975 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:56.783246 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:56.783080 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:57.143176 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:57.143093 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="d977236251767dab5ec8c8010360b95913371831643334b367976e0bc72dbf67" exitCode=0 Jul 11 23:31:57.143311 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:57.143170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"d977236251767dab5ec8c8010360b95913371831643334b367976e0bc72dbf67"} Jul 11 23:31:57.782352 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:57.782311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:57.782588 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:57.782440 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:31:58.148496 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:58.148259 2580 generic.go:356] "Generic (PLEG): container finished" podID="77b79610-2e35-4d57-bca6-5ac0ee3d0b2f" containerID="5a0e506dde47d0b848e81feb74470b16f1d8ad3ea61ca5c8a9f8c75784869ff4" exitCode=0 Jul 11 23:31:58.148496 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:58.148295 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerDied","Data":"5a0e506dde47d0b848e81feb74470b16f1d8ad3ea61ca5c8a9f8c75784869ff4"} Jul 11 23:31:58.782338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:58.782312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:31:58.782535 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:58.782321 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:31:58.782535 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:58.782409 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:31:58.782535 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:58.782504 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:31:59.154092 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:59.154063 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" event={"ID":"77b79610-2e35-4d57-bca6-5ac0ee3d0b2f","Type":"ContainerStarted","Data":"e4cca104cdd5398abcccb8bdff617b66bfc30bc4585fb1088c3b85ba76c9bcec"} Jul 11 23:31:59.175230 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:59.175182 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-mbh6w" podStartSLOduration=6.871331528 podStartE2EDuration="38.175165718s" totalImagesPullingTime="31.30383419s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:31:59.174991452 +0000 UTC m=+38.926739018" watchObservedRunningTime="2026-07-11 23:31:59.175165718 +0000 UTC m=+38.926913220" Jul 11 23:31:59.782264 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:31:59.782236 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:31:59.782443 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:31:59.782351 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:00.783827 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:00.783795 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:00.784171 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:00.783873 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:00.784171 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:00.783900 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:00.784171 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:00.783923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:01.782513 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:01.782462 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:01.782686 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:01.782591 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:02.782313 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:02.782271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:02.782313 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:02.782271 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:02.782807 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:02.782435 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:02.782807 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:02.782441 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:03.782010 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:03.781974 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:03.782211 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:03.782084 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:04.782608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:04.782574 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:04.783027 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:04.782710 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:04.783027 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:04.782763 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:04.783027 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:04.782796 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:05.782443 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:05.782415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:05.782703 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:05.782522 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:06.782282 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:06.782252 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:06.782497 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:06.782372 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:06.782497 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:06.782441 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:06.782637 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:06.782540 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:07.782543 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:07.782511 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:07.782916 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:07.782625 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:08.782859 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:08.782823 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:08.783319 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:08.782828 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:08.783319 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:08.782953 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:08.783319 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:08.782994 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:09.782974 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:09.782937 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:09.783364 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:09.783048 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:10.783371 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:10.783332 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:10.783846 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:10.783435 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:10.783846 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:10.783493 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:10.783846 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:10.783579 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:11.782539 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:11.782502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:11.782740 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:11.782637 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:12.782589 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:12.782555 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:12.782960 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:12.782552 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:12.782960 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:12.782658 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:12.782960 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:12.782784 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:13.782068 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:13.782040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:13.782285 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:13.782145 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cdt7s" podUID="338bdff5-9113-462b-a1db-947024a1db1d" Jul 11 23:32:14.782702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:14.782668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:14.783170 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:14.782794 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gstql" podUID="f5de7bc3-7a27-4033-9303-380b33e254f9" Jul 11 23:32:14.783170 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:14.782841 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:14.783170 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:14.782923 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-b465v" podUID="0d21d616-2e36-419b-945e-03b79b94946d" Jul 11 23:32:15.605249 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.605223 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-140-223.ec2.internal" event="NodeReady" Jul 11 23:32:15.605414 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.605339 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 11 23:32:15.649269 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.649239 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-qxk48"] Jul 11 23:32:15.670483 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.670435 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-ngdqs"] Jul 11 23:32:15.670625 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.670560 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.672841 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.672813 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 11 23:32:15.673048 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.673034 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-8jgcr\"" Jul 11 23:32:15.673148 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.673133 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 11 23:32:15.688493 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.688449 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qxk48"] Jul 11 23:32:15.688604 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.688501 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ngdqs"] Jul 11 23:32:15.688604 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.688571 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.691412 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.691393 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 11 23:32:15.691540 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.691453 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 11 23:32:15.691608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.691546 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-8s5q6\"" Jul 11 23:32:15.691608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.691560 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 11 23:32:15.753774 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.753742 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-vt9sd"] Jul 11 23:32:15.769778 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769739 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmr82\" (UniqueName: \"kubernetes.io/projected/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-kube-api-access-mmr82\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.769938 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769790 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-tmp-dir\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.769938 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769829 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-metrics-tls\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.769938 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769920 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-config-volume\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.770067 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769953 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djtg6\" (UniqueName: \"kubernetes.io/projected/9529b946-8c65-47e9-90a3-1f33d9188cf1-kube-api-access-djtg6\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.770067 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.769988 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9529b946-8c65-47e9-90a3-1f33d9188cf1-cert\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.771603 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.771585 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vt9sd"] Jul 11 23:32:15.771730 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.771713 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.774131 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.774097 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-k95xv\"" Jul 11 23:32:15.774131 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.774112 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 11 23:32:15.774296 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.774134 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 11 23:32:15.774296 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.774187 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 11 23:32:15.774421 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.774397 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 11 23:32:15.782735 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.782716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:15.784944 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.784928 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nxtg4\"" Jul 11 23:32:15.785043 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.784934 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 11 23:32:15.870366 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-config-volume\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.870366 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-djtg6\" (UniqueName: \"kubernetes.io/projected/9529b946-8c65-47e9-90a3-1f33d9188cf1-kube-api-access-djtg6\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.870366 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8c2819-ee75-40b5-ade5-328096d0c06a-crio-socket\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870371 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9529b946-8c65-47e9-90a3-1f33d9188cf1-cert\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870413 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p77gq\" (UniqueName: \"kubernetes.io/projected/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-api-access-p77gq\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mmr82\" (UniqueName: \"kubernetes.io/projected/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-kube-api-access-mmr82\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-tmp-dir\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8c2819-ee75-40b5-ade5-328096d0c06a-data-volume\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870555 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8c2819-ee75-40b5-ade5-328096d0c06a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.870658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870596 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-metrics-tls\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.871019 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870806 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.871019 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870870 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-tmp-dir\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.871019 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.870993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-config-volume\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.874266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.874240 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-metrics-tls\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.874575 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.874557 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9529b946-8c65-47e9-90a3-1f33d9188cf1-cert\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.878255 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.878231 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmr82\" (UniqueName: \"kubernetes.io/projected/f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b-kube-api-access-mmr82\") pod \"dns-default-qxk48\" (UID: \"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b\") " pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.878513 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.878492 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-djtg6\" (UniqueName: \"kubernetes.io/projected/9529b946-8c65-47e9-90a3-1f33d9188cf1-kube-api-access-djtg6\") pod \"ingress-canary-ngdqs\" (UID: \"9529b946-8c65-47e9-90a3-1f33d9188cf1\") " pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:15.971536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971500 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8c2819-ee75-40b5-ade5-328096d0c06a-data-volume\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971537 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8c2819-ee75-40b5-ade5-328096d0c06a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971753 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971668 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971753 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8c2819-ee75-40b5-ade5-328096d0c06a-crio-socket\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971753 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971742 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p77gq\" (UniqueName: \"kubernetes.io/projected/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-api-access-p77gq\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971911 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971815 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/9d8c2819-ee75-40b5-ade5-328096d0c06a-crio-socket\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.971911 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.971849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/9d8c2819-ee75-40b5-ade5-328096d0c06a-data-volume\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.972219 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.972197 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.973750 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.973734 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/9d8c2819-ee75-40b5-ade5-328096d0c06a-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.979061 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.979042 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p77gq\" (UniqueName: \"kubernetes.io/projected/9d8c2819-ee75-40b5-ade5-328096d0c06a-kube-api-access-p77gq\") pod \"insights-runtime-extractor-vt9sd\" (UID: \"9d8c2819-ee75-40b5-ade5-328096d0c06a\") " pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:15.979918 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.979903 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:15.997032 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:15.997011 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-ngdqs" Jul 11 23:32:16.080547 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.080510 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-vt9sd" Jul 11 23:32:16.188930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.188906 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-qxk48"] Jul 11 23:32:16.192901 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:16.192870 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2dec4fb_b5f0_408a_aad5_b94ec6b85e5b.slice/crio-75f6f82237dbef2277a46b1b7c7175694ab565910b0b4af48ab457454c6a7a54 WatchSource:0}: Error finding container 75f6f82237dbef2277a46b1b7c7175694ab565910b0b4af48ab457454c6a7a54: Status 404 returned error can't find the container with id 75f6f82237dbef2277a46b1b7c7175694ab565910b0b4af48ab457454c6a7a54 Jul 11 23:32:16.212205 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.212135 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-ngdqs"] Jul 11 23:32:16.216286 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:16.216253 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9529b946_8c65_47e9_90a3_1f33d9188cf1.slice/crio-8d0a934d0b612e20f7dd9f2f924eebb414d8270f28ac08e4b9a65c7be410493d WatchSource:0}: Error finding container 8d0a934d0b612e20f7dd9f2f924eebb414d8270f28ac08e4b9a65c7be410493d: Status 404 returned error can't find the container with id 8d0a934d0b612e20f7dd9f2f924eebb414d8270f28ac08e4b9a65c7be410493d Jul 11 23:32:16.233927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.233900 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-vt9sd"] Jul 11 23:32:16.236694 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:16.236672 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d8c2819_ee75_40b5_ade5_328096d0c06a.slice/crio-14fc9c78fc33811a59f0658519b85146c0b8b309ee9eb046698d8ba40f190d32 WatchSource:0}: Error finding container 14fc9c78fc33811a59f0658519b85146c0b8b309ee9eb046698d8ba40f190d32: Status 404 returned error can't find the container with id 14fc9c78fc33811a59f0658519b85146c0b8b309ee9eb046698d8ba40f190d32 Jul 11 23:32:16.782753 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.782720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:16.783301 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.782720 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:16.785627 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.785206 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 11 23:32:16.786028 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.786008 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 11 23:32:16.786532 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.786196 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qb5vv\"" Jul 11 23:32:16.786532 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:16.786271 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 11 23:32:17.198028 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:17.197881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vt9sd" event={"ID":"9d8c2819-ee75-40b5-ade5-328096d0c06a","Type":"ContainerStarted","Data":"14fc9c78fc33811a59f0658519b85146c0b8b309ee9eb046698d8ba40f190d32"} Jul 11 23:32:17.200598 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:17.200505 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ngdqs" event={"ID":"9529b946-8c65-47e9-90a3-1f33d9188cf1","Type":"ContainerStarted","Data":"8d0a934d0b612e20f7dd9f2f924eebb414d8270f28ac08e4b9a65c7be410493d"} Jul 11 23:32:17.202655 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:17.202597 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxk48" event={"ID":"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b","Type":"ContainerStarted","Data":"75f6f82237dbef2277a46b1b7c7175694ab565910b0b4af48ab457454c6a7a54"} Jul 11 23:32:19.291028 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.290993 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q"] Jul 11 23:32:19.301601 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.301502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q"] Jul 11 23:32:19.301772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.301648 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:19.304773 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.304538 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 11 23:32:19.304773 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.304544 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-rbg92\"" Jul 11 23:32:19.403166 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.403125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed966a69-ddc7-4d00-818b-fe13932813dc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtn6q\" (UID: \"ed966a69-ddc7-4d00-818b-fe13932813dc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:19.503713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.503675 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed966a69-ddc7-4d00-818b-fe13932813dc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtn6q\" (UID: \"ed966a69-ddc7-4d00-818b-fe13932813dc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:19.506814 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.506785 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/ed966a69-ddc7-4d00-818b-fe13932813dc-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-gtn6q\" (UID: \"ed966a69-ddc7-4d00-818b-fe13932813dc\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:19.613181 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.613087 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:19.779992 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.779957 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:32:19.782479 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.782450 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.785325 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785192 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 11 23:32:19.785325 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785192 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 11 23:32:19.785325 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785247 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 11 23:32:19.785325 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785302 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 11 23:32:19.785612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785366 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 11 23:32:19.785612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785446 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 11 23:32:19.785612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785544 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 11 23:32:19.785612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.785586 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-nt8tb\"" Jul 11 23:32:19.793552 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.793525 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:32:19.806885 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.806862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.807073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.806908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.807073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.807014 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.807073 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.807054 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.807222 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.807083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.807222 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.807147 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsdvt\" (UniqueName: \"kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908509 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908405 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lsdvt\" (UniqueName: \"kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908509 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908509 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908506 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908756 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908550 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908756 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908577 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.908756 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.908601 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.909291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.909267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.909407 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.909318 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.909485 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.909405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.911147 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.911126 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.911546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.911528 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:19.916717 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:19.916663 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsdvt\" (UniqueName: \"kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt\") pod \"console-664494f996-kgqwr\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:20.093782 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:20.093738 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:21.134141 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:21.134103 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-dwvq7" Jul 11 23:32:21.530338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:21.528731 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:32:21.534359 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:21.534308 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d7f7c30_6c08_4610_8f2a_04cd4b8975d5.slice/crio-365b20b9cae7ef385a6187382b269cc2d71ef7815b30e59d5e70d0757158a23a WatchSource:0}: Error finding container 365b20b9cae7ef385a6187382b269cc2d71ef7815b30e59d5e70d0757158a23a: Status 404 returned error can't find the container with id 365b20b9cae7ef385a6187382b269cc2d71ef7815b30e59d5e70d0757158a23a Jul 11 23:32:21.551877 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:21.551849 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q"] Jul 11 23:32:21.557252 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:21.557207 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poded966a69_ddc7_4d00_818b_fe13932813dc.slice/crio-a24807542f80e76723083e4646147bf4f5b96dc7b6293791e7c946d55ebc36a0 WatchSource:0}: Error finding container a24807542f80e76723083e4646147bf4f5b96dc7b6293791e7c946d55ebc36a0: Status 404 returned error can't find the container with id a24807542f80e76723083e4646147bf4f5b96dc7b6293791e7c946d55ebc36a0 Jul 11 23:32:22.222095 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.222058 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vt9sd" event={"ID":"9d8c2819-ee75-40b5-ade5-328096d0c06a","Type":"ContainerStarted","Data":"05decc2f17e67716f1791cfa0c7949047ec478b6014c02ec111a3b0e5f6d4342"} Jul 11 23:32:22.222586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.222166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vt9sd" event={"ID":"9d8c2819-ee75-40b5-ade5-328096d0c06a","Type":"ContainerStarted","Data":"0a15f6fbf293c1eb5b361b5d2715c26f013062c0e5eae1b4b2d3e2db0015224f"} Jul 11 23:32:22.223884 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.223853 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-ngdqs" event={"ID":"9529b946-8c65-47e9-90a3-1f33d9188cf1","Type":"ContainerStarted","Data":"13542dc5e7bf5f8c7a414daac60f1ee39d7e4f1f0e12c1e8f3e3ca1627a389db"} Jul 11 23:32:22.225988 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.225962 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxk48" event={"ID":"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b","Type":"ContainerStarted","Data":"13d1119184646194f05752f4d20b1b18a5addaf092309037b5e079dae5aaec3a"} Jul 11 23:32:22.226100 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.226010 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-qxk48" event={"ID":"f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b","Type":"ContainerStarted","Data":"d616d7ae19670539c2b909d1521c8673508aac59a0804063382f40d759f9f6bb"} Jul 11 23:32:22.226100 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.226068 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:22.227439 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.227416 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" event={"ID":"ed966a69-ddc7-4d00-818b-fe13932813dc","Type":"ContainerStarted","Data":"a24807542f80e76723083e4646147bf4f5b96dc7b6293791e7c946d55ebc36a0"} Jul 11 23:32:22.228713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.228686 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664494f996-kgqwr" event={"ID":"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5","Type":"ContainerStarted","Data":"365b20b9cae7ef385a6187382b269cc2d71ef7815b30e59d5e70d0757158a23a"} Jul 11 23:32:22.237397 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.237354 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-ngdqs" podStartSLOduration=2.082275961 podStartE2EDuration="7.237342704s" totalImagesPullingTime="5.155066743s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:22.236895144 +0000 UTC m=+61.988642646" watchObservedRunningTime="2026-07-11 23:32:22.237342704 +0000 UTC m=+61.989090206" Jul 11 23:32:22.252062 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:22.251664 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-qxk48" podStartSLOduration=2.073432797 podStartE2EDuration="7.251645595s" totalImagesPullingTime="5.178212798s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:22.251036467 +0000 UTC m=+62.002784010" watchObservedRunningTime="2026-07-11 23:32:22.251645595 +0000 UTC m=+62.003393096" Jul 11 23:32:23.234842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:23.234768 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" event={"ID":"ed966a69-ddc7-4d00-818b-fe13932813dc","Type":"ContainerStarted","Data":"4ec8823347e8121b9372ec801c297a2146d3a191d0b3a078c9fac150cccab770"} Jul 11 23:32:23.235808 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:23.235771 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:23.242169 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:23.242142 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" Jul 11 23:32:23.249856 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:23.249757 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-gtn6q" podStartSLOduration=3.095305968 podStartE2EDuration="4.249739614s" totalImagesPullingTime="1.154433646s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:23.249236041 +0000 UTC m=+63.000983545" watchObservedRunningTime="2026-07-11 23:32:23.249739614 +0000 UTC m=+63.001487117" Jul 11 23:32:24.357905 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.357874 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-4qgln"] Jul 11 23:32:24.362691 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.362658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.365763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.365730 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 11 23:32:24.366583 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.366524 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 11 23:32:24.366583 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.366569 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 11 23:32:24.366781 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.366615 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 11 23:32:24.366781 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.366628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-5zdwh\"" Jul 11 23:32:24.366781 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.366746 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 11 23:32:24.380647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.380610 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-4qgln"] Jul 11 23:32:24.445712 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.445679 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.445712 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.445717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.445948 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.445792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f41fe7-4cd6-496a-b762-132e803f1af7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.445948 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.445822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s5528\" (UniqueName: \"kubernetes.io/projected/e0f41fe7-4cd6-496a-b762-132e803f1af7-kube-api-access-s5528\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.546732 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.546693 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.546732 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.546736 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.546955 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:24.546874 2580 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jul 11 23:32:24.546955 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:24.546954 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls podName:e0f41fe7-4cd6-496a-b762-132e803f1af7 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:25.046928823 +0000 UTC m=+64.798676308 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls") pod "prometheus-operator-64785b6675-4qgln" (UID: "e0f41fe7-4cd6-496a-b762-132e803f1af7") : secret "prometheus-operator-tls" not found Jul 11 23:32:24.547061 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.547041 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f41fe7-4cd6-496a-b762-132e803f1af7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.547113 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.547075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s5528\" (UniqueName: \"kubernetes.io/projected/e0f41fe7-4cd6-496a-b762-132e803f1af7-kube-api-access-s5528\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.547799 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.547773 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e0f41fe7-4cd6-496a-b762-132e803f1af7-metrics-client-ca\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.549585 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.549558 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:24.558246 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:24.558218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s5528\" (UniqueName: \"kubernetes.io/projected/e0f41fe7-4cd6-496a-b762-132e803f1af7-kube-api-access-s5528\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:25.050760 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.050716 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:25.053425 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.053393 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/e0f41fe7-4cd6-496a-b762-132e803f1af7-prometheus-operator-tls\") pod \"prometheus-operator-64785b6675-4qgln\" (UID: \"e0f41fe7-4cd6-496a-b762-132e803f1af7\") " pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:25.274628 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.274578 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" Jul 11 23:32:25.554582 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.554538 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:25.555150 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.554612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:25.556978 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.556952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 11 23:32:25.557091 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.556952 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 11 23:32:25.568103 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.568071 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/338bdff5-9113-462b-a1db-947024a1db1d-metrics-certs\") pod \"network-metrics-daemon-cdt7s\" (UID: \"338bdff5-9113-462b-a1db-947024a1db1d\") " pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:25.568281 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.568121 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/f5de7bc3-7a27-4033-9303-380b33e254f9-original-pull-secret\") pod \"global-pull-secret-syncer-gstql\" (UID: \"f5de7bc3-7a27-4033-9303-380b33e254f9\") " pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:25.655627 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.655593 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:25.658278 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.658254 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 11 23:32:25.668832 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.668798 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 11 23:32:25.678952 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.678922 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9l67\" (UniqueName: \"kubernetes.io/projected/0d21d616-2e36-419b-945e-03b79b94946d-kube-api-access-m9l67\") pod \"network-check-target-b465v\" (UID: \"0d21d616-2e36-419b-945e-03b79b94946d\") " pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:25.692734 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.692705 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-nxtg4\"" Jul 11 23:32:25.701489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.701452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cdt7s" Jul 11 23:32:25.796790 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.796764 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gstql" Jul 11 23:32:25.807659 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.807628 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-qb5vv\"" Jul 11 23:32:25.815758 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.815726 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:25.872742 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.869818 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-64785b6675-4qgln"] Jul 11 23:32:25.882512 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:25.882449 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode0f41fe7_4cd6_496a_b762_132e803f1af7.slice/crio-65e879576f444e42d87ffae64d4f15e51347c58c6d86849562fca5f04a4b5f7c WatchSource:0}: Error finding container 65e879576f444e42d87ffae64d4f15e51347c58c6d86849562fca5f04a4b5f7c: Status 404 returned error can't find the container with id 65e879576f444e42d87ffae64d4f15e51347c58c6d86849562fca5f04a4b5f7c Jul 11 23:32:25.885874 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.885577 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cdt7s"] Jul 11 23:32:25.892872 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:25.892780 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod338bdff5_9113_462b_a1db_947024a1db1d.slice/crio-ea12957821c6164f877919132c47d76993b05b3380d636a54e8958c81e33b76a WatchSource:0}: Error finding container ea12957821c6164f877919132c47d76993b05b3380d636a54e8958c81e33b76a: Status 404 returned error can't find the container with id ea12957821c6164f877919132c47d76993b05b3380d636a54e8958c81e33b76a Jul 11 23:32:25.950981 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.950949 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gstql"] Jul 11 23:32:25.954440 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:25.954411 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5de7bc3_7a27_4033_9303_380b33e254f9.slice/crio-78113c8125d13452189ac08decdbfcc60dea639f80e5394e80d4e28644ca04d5 WatchSource:0}: Error finding container 78113c8125d13452189ac08decdbfcc60dea639f80e5394e80d4e28644ca04d5: Status 404 returned error can't find the container with id 78113c8125d13452189ac08decdbfcc60dea639f80e5394e80d4e28644ca04d5 Jul 11 23:32:25.972499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:25.972457 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-b465v"] Jul 11 23:32:25.975789 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:25.975760 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d21d616_2e36_419b_945e_03b79b94946d.slice/crio-65981139a7223e73ad30eb2b0ecf365d855be3c8e609071eddd931398621309c WatchSource:0}: Error finding container 65981139a7223e73ad30eb2b0ecf365d855be3c8e609071eddd931398621309c: Status 404 returned error can't find the container with id 65981139a7223e73ad30eb2b0ecf365d855be3c8e609071eddd931398621309c Jul 11 23:32:26.247722 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.247687 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b465v" event={"ID":"0d21d616-2e36-419b-945e-03b79b94946d","Type":"ContainerStarted","Data":"65981139a7223e73ad30eb2b0ecf365d855be3c8e609071eddd931398621309c"} Jul 11 23:32:26.248863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.248839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" event={"ID":"e0f41fe7-4cd6-496a-b762-132e803f1af7","Type":"ContainerStarted","Data":"65e879576f444e42d87ffae64d4f15e51347c58c6d86849562fca5f04a4b5f7c"} Jul 11 23:32:26.250926 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.250898 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-vt9sd" event={"ID":"9d8c2819-ee75-40b5-ade5-328096d0c06a","Type":"ContainerStarted","Data":"d4ba2482cfceb0cad57dc459a323515b00e623d4defb292a43b8e92575aa6aac"} Jul 11 23:32:26.251960 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.251938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gstql" event={"ID":"f5de7bc3-7a27-4033-9303-380b33e254f9","Type":"ContainerStarted","Data":"78113c8125d13452189ac08decdbfcc60dea639f80e5394e80d4e28644ca04d5"} Jul 11 23:32:26.253503 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.253457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664494f996-kgqwr" event={"ID":"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5","Type":"ContainerStarted","Data":"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d"} Jul 11 23:32:26.254745 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.254722 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdt7s" event={"ID":"338bdff5-9113-462b-a1db-947024a1db1d","Type":"ContainerStarted","Data":"ea12957821c6164f877919132c47d76993b05b3380d636a54e8958c81e33b76a"} Jul 11 23:32:26.269225 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.269169 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-vt9sd" podStartSLOduration=1.94217554 podStartE2EDuration="11.269155971s" totalImagesPullingTime="9.326980431s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:32:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:26.267653363 +0000 UTC m=+66.019400893" watchObservedRunningTime="2026-07-11 23:32:26.269155971 +0000 UTC m=+66.020903472" Jul 11 23:32:26.282503 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:26.282436 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-664494f996-kgqwr" podStartSLOduration=3.102013455 podStartE2EDuration="7.282419707s" totalImagesPullingTime="4.180406252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:26.281856445 +0000 UTC m=+66.033603949" watchObservedRunningTime="2026-07-11 23:32:26.282419707 +0000 UTC m=+66.034167213" Jul 11 23:32:28.164080 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.164048 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:32:28.165944 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.165921 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.175069 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.175040 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 11 23:32:28.177212 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.177185 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:32:28.265136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.265089 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdt7s" event={"ID":"338bdff5-9113-462b-a1db-947024a1db1d","Type":"ContainerStarted","Data":"52f945d43966111915f788aeb1c0fb06388f0582341524741ea1cb55b724eac1"} Jul 11 23:32:28.265321 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.265160 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cdt7s" event={"ID":"338bdff5-9113-462b-a1db-947024a1db1d","Type":"ContainerStarted","Data":"080a71823006176690721b06ff341734d88ce3f3636e2ed9af22d381955b1054"} Jul 11 23:32:28.267464 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.267434 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" event={"ID":"e0f41fe7-4cd6-496a-b762-132e803f1af7","Type":"ContainerStarted","Data":"836c35ba886132d0efc780f26beb55ea07b4e4e397ef55c21f1e72e02d8299ce"} Jul 11 23:32:28.267585 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.267507 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" event={"ID":"e0f41fe7-4cd6-496a-b762-132e803f1af7","Type":"ContainerStarted","Data":"2524a6cfc7c03d5e2a7d1b20cc4174eab2c045043cfea59171ea7ead2eb92457"} Jul 11 23:32:28.277127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277033 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm6d4\" (UniqueName: \"kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277125 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277440 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277150 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277440 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277440 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277207 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.277440 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.277332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.280145 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.279751 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cdt7s" podStartSLOduration=65.843796821 podStartE2EDuration="1m7.279732576s" totalImagesPullingTime="1.435935755s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:28.279234633 +0000 UTC m=+68.030982146" watchObservedRunningTime="2026-07-11 23:32:28.279732576 +0000 UTC m=+68.031480084" Jul 11 23:32:28.299884 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.299824 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-64785b6675-4qgln" podStartSLOduration=2.853623119 podStartE2EDuration="4.299806736s" totalImagesPullingTime="1.446183617s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:28.298731666 +0000 UTC m=+68.050479169" watchObservedRunningTime="2026-07-11 23:32:28.299806736 +0000 UTC m=+68.051554240" Jul 11 23:32:28.378424 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378486 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378512 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378541 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm6d4\" (UniqueName: \"kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378590 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378621 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.378880 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.378650 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.379416 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.379306 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.379416 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.379363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.379416 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.379363 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.379646 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.379574 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.382351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.382325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.383088 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.383044 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.389635 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.389592 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm6d4\" (UniqueName: \"kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4\") pod \"console-58b5584696-rfvcm\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:28.477645 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:28.477572 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:29.726517 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.726480 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j"] Jul 11 23:32:29.742996 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.742966 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j"] Jul 11 23:32:29.743181 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.743094 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.747684 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.747550 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-4mhpk\"" Jul 11 23:32:29.747684 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.747573 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 11 23:32:29.748487 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.748451 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 11 23:32:29.773715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.773683 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-qt64h"] Jul 11 23:32:29.791606 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.791574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02312fb7-9694-4563-b92f-f2584c72ef7e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.791606 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.791611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.791838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.791648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.791838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.791701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2bt82\" (UniqueName: \"kubernetes.io/projected/02312fb7-9694-4563-b92f-f2584c72ef7e-kube-api-access-2bt82\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.794569 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.794539 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wlqh4"] Jul 11 23:32:29.794682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.794648 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.797434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.797411 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 11 23:32:29.797553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.797439 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 11 23:32:29.797553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.797505 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-j7gjb\"" Jul 11 23:32:29.798091 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.798074 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 11 23:32:29.814650 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.814617 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wlqh4"] Jul 11 23:32:29.814823 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.814758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.819263 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.819231 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 11 23:32:29.819414 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.819333 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 11 23:32:29.819414 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.819370 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-lwhj9\"" Jul 11 23:32:29.819556 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.819543 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 11 23:32:29.892670 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892625 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.892670 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892665 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892688 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-textfile\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892708 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-metrics-client-ca\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hhnpm\" (UniqueName: \"kubernetes.io/projected/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-api-access-hhnpm\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892792 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-sys\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892818 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-wtmp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.892915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892892 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892929 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02312fb7-9694-4563-b92f-f2584c72ef7e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892954 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.892975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9pnnp\" (UniqueName: \"kubernetes.io/projected/a0fc3079-83c6-4d06-bed2-d73eb17109fa-kube-api-access-9pnnp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893056 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893081 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893102 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2bt82\" (UniqueName: \"kubernetes.io/projected/02312fb7-9694-4563-b92f-f2584c72ef7e-kube-api-access-2bt82\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.893112 2580 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-root\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.893306 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.893255 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls podName:02312fb7-9694-4563-b92f-f2584c72ef7e nodeName:}" failed. No retries permitted until 2026-07-11 23:32:30.393231322 +0000 UTC m=+70.144978812 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-lc77j" (UID: "02312fb7-9694-4563-b92f-f2584c72ef7e") : secret "openshift-state-metrics-tls" not found Jul 11 23:32:29.893895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.893683 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/02312fb7-9694-4563-b92f-f2584c72ef7e-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.895810 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.895776 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.901753 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.901718 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2bt82\" (UniqueName: \"kubernetes.io/projected/02312fb7-9694-4563-b92f-f2584c72ef7e-kube-api-access-2bt82\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:29.993817 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.993721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.993817 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.993775 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9pnnp\" (UniqueName: \"kubernetes.io/projected/a0fc3079-83c6-4d06-bed2-d73eb17109fa-kube-api-access-9pnnp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994034 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.993902 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994034 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.993957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-root\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994034 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-root\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994202 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.994036 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 11 23:32:29.994202 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994202 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.994092 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls podName:a0fc3079-83c6-4d06-bed2-d73eb17109fa nodeName:}" failed. No retries permitted until 2026-07-11 23:32:30.494073213 +0000 UTC m=+70.245820701 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls") pod "node-exporter-qt64h" (UID: "a0fc3079-83c6-4d06-bed2-d73eb17109fa") : secret "node-exporter-tls" not found Jul 11 23:32:29.994202 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994155 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994202 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994189 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-textfile\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-metrics-client-ca\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994240 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hhnpm\" (UniqueName: \"kubernetes.io/projected/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-api-access-hhnpm\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-sys\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994290 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-wtmp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994309 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994324 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994341 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994360 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.994447 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994443 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-accelerators-collector-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994903 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994490 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-sys\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.994552 2580 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 11 23:32:29.994903 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994564 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-wtmp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.994903 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:29.994613 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls podName:e008a97f-3dfd-4ca0-b4fb-14717a72f1b8 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:30.49459763 +0000 UTC m=+70.246345126 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls") pod "kube-state-metrics-785778ff74-wlqh4" (UID: "e008a97f-3dfd-4ca0-b4fb-14717a72f1b8") : secret "kube-state-metrics-tls" not found Jul 11 23:32:29.994903 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994859 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-textfile\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.995165 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.994943 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-volume-directive-shadow\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.995231 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.995182 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-metrics-client-ca\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.995231 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.995204 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.995631 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.995606 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/a0fc3079-83c6-4d06-bed2-d73eb17109fa-metrics-client-ca\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:29.997017 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.996987 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:29.997357 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:29.997325 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:30.002629 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.002604 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9pnnp\" (UniqueName: \"kubernetes.io/projected/a0fc3079-83c6-4d06-bed2-d73eb17109fa-kube-api-access-9pnnp\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:30.002972 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.002947 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hhnpm\" (UniqueName: \"kubernetes.io/projected/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-api-access-hhnpm\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:30.093895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.093852 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:30.094068 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.093920 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:30.099503 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.099455 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:30.297098 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.297026 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:32:30.398300 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.398258 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:30.401019 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.400993 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/02312fb7-9694-4563-b92f-f2584c72ef7e-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-lc77j\" (UID: \"02312fb7-9694-4563-b92f-f2584c72ef7e\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:30.499627 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.499586 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:30.499817 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.499662 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:30.499817 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:30.499789 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 11 23:32:30.499982 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:30.499858 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls podName:a0fc3079-83c6-4d06-bed2-d73eb17109fa nodeName:}" failed. No retries permitted until 2026-07-11 23:32:31.499838918 +0000 UTC m=+71.251586399 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls") pod "node-exporter-qt64h" (UID: "a0fc3079-83c6-4d06-bed2-d73eb17109fa") : secret "node-exporter-tls" not found Jul 11 23:32:30.502409 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.502379 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/e008a97f-3dfd-4ca0-b4fb-14717a72f1b8-kube-state-metrics-tls\") pod \"kube-state-metrics-785778ff74-wlqh4\" (UID: \"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8\") " pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:30.654905 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.654820 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" Jul 11 23:32:30.725689 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.725658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" Jul 11 23:32:30.805481 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.805430 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:32:30.819085 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.819050 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.821570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821544 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 11 23:32:30.821727 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821622 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 11 23:32:30.821727 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821649 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 11 23:32:30.821727 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821624 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 11 23:32:30.821886 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821791 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-27w9l\"" Jul 11 23:32:30.821993 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.821976 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 11 23:32:30.822216 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.822200 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 11 23:32:30.822410 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.822392 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 11 23:32:30.822410 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.822402 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 11 23:32:30.822582 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.822430 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 11 23:32:30.839415 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.837614 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:32:30.904422 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904446 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5c6ss\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904513 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904570 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904546 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904645 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904681 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904710 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904741 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904770 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:30.904815 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:30.904802 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005782 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005936 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005793 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005936 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005831 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005936 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005936 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005892 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.005936 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5c6ss\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006149 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006149 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:31.005976 2580 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 11 23:32:31.006149 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:31.006062 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls podName:476b9d8d-5e3b-4492-b83b-eda3c11f3091 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:31.506040717 +0000 UTC m=+71.257788201 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091") : secret "alertmanager-main-tls" not found Jul 11 23:32:31.006149 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.005983 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006272 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006279 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.006316 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.006725 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:31.006522 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle podName:476b9d8d-5e3b-4492-b83b-eda3c11f3091 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:31.50650398 +0000 UTC m=+71.258251474 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091") : configmap references non-existent config key: ca-bundle.crt Jul 11 23:32:31.007097 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.007068 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.008943 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.008849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009099 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009078 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009109 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009259 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009231 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009335 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009277 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009335 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009283 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009601 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.009671 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.009653 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.018329 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.018307 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5c6ss\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.305724 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.304991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j"] Jul 11 23:32:31.307481 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:31.307429 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02312fb7_9694_4563_b92f_f2584c72ef7e.slice/crio-53b108f04419a88df71816a7f797d7e36f7df11f9c34c04231df41e7ed3e19df WatchSource:0}: Error finding container 53b108f04419a88df71816a7f797d7e36f7df11f9c34c04231df41e7ed3e19df: Status 404 returned error can't find the container with id 53b108f04419a88df71816a7f797d7e36f7df11f9c34c04231df41e7ed3e19df Jul 11 23:32:31.308389 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.308299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-b465v" event={"ID":"0d21d616-2e36-419b-945e-03b79b94946d","Type":"ContainerStarted","Data":"ac092ea59662bbea55f0c2a1cbfde8ee0624c71b8c8193991fba139e11f85e50"} Jul 11 23:32:31.308389 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.308375 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:32:31.323020 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.322976 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-b465v" podStartSLOduration=65.159134244 podStartE2EDuration="1m10.322959613s" totalImagesPullingTime="5.163825369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:31.322034046 +0000 UTC m=+71.073781550" watchObservedRunningTime="2026-07-11 23:32:31.322959613 +0000 UTC m=+71.074707143" Jul 11 23:32:31.510924 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.510895 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:31.511103 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.510931 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.511103 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.510977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.511931 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.511868 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.513242 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.513221 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/a0fc3079-83c6-4d06-bed2-d73eb17109fa-node-exporter-tls\") pod \"node-exporter-qt64h\" (UID: \"a0fc3079-83c6-4d06-bed2-d73eb17109fa\") " pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:31.513484 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.513450 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.533805 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.533757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:32:31.535115 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.535093 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-785778ff74-wlqh4"] Jul 11 23:32:31.536721 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:31.536696 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode008a97f_3dfd_4ca0_b4fb_14717a72f1b8.slice/crio-f0682c9b12925c76e5699e640fe400ed4fca5ed827548cf169ee23c100d4532c WatchSource:0}: Error finding container f0682c9b12925c76e5699e640fe400ed4fca5ed827548cf169ee23c100d4532c: Status 404 returned error can't find the container with id f0682c9b12925c76e5699e640fe400ed4fca5ed827548cf169ee23c100d4532c Jul 11 23:32:31.537180 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:31.537158 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9cb5d34a_12a8_476c_8673_625595bdb724.slice/crio-e4989abeadaece6a9e7a63e006d3d8a2d16ce4dee9ea191cb735e7e9b0d9ebd6 WatchSource:0}: Error finding container e4989abeadaece6a9e7a63e006d3d8a2d16ce4dee9ea191cb735e7e9b0d9ebd6: Status 404 returned error can't find the container with id e4989abeadaece6a9e7a63e006d3d8a2d16ce4dee9ea191cb735e7e9b0d9ebd6 Jul 11 23:32:31.606582 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.606551 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-qt64h" Jul 11 23:32:31.619073 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:31.619032 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fc3079_83c6_4d06_bed2_d73eb17109fa.slice/crio-f9e7ae60b59369b05481a9df42f7f5156db2519c562b4e6af59b1674f35995a0 WatchSource:0}: Error finding container f9e7ae60b59369b05481a9df42f7f5156db2519c562b4e6af59b1674f35995a0: Status 404 returned error can't find the container with id f9e7ae60b59369b05481a9df42f7f5156db2519c562b4e6af59b1674f35995a0 Jul 11 23:32:31.712366 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.712333 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-668c9bc467-7kczj"] Jul 11 23:32:31.715352 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.715334 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.718161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718118 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-a9155id3cf0tl\"" Jul 11 23:32:31.718161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718136 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 11 23:32:31.718161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718143 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 11 23:32:31.718449 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718169 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 11 23:32:31.718449 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718176 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-mpkgn\"" Jul 11 23:32:31.718449 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718275 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 11 23:32:31.718449 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.718404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 11 23:32:31.726779 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.726756 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668c9bc467-7kczj"] Jul 11 23:32:31.729072 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.729050 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:32:31.813069 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813031 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-grpc-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813083 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76f110e1-a005-49f5-ad46-ea8831fd7440-metrics-client-ca\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813201 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mql82\" (UniqueName: \"kubernetes.io/projected/76f110e1-a005-49f5-ad46-ea8831fd7440-kube-api-access-mql82\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813273 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813297 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813334 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.813489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.813371 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.865056 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.865018 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:32:31.868820 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:31.868788 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod476b9d8d_5e3b_4492_b83b_eda3c11f3091.slice/crio-3482791d1fed3610b62aacb55450b8601747cc76d48180b786485f95403b4b82 WatchSource:0}: Error finding container 3482791d1fed3610b62aacb55450b8601747cc76d48180b786485f95403b4b82: Status 404 returned error can't find the container with id 3482791d1fed3610b62aacb55450b8601747cc76d48180b786485f95403b4b82 Jul 11 23:32:31.914077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914038 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-grpc-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914243 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914121 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76f110e1-a005-49f5-ad46-ea8831fd7440-metrics-client-ca\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914243 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914243 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mql82\" (UniqueName: \"kubernetes.io/projected/76f110e1-a005-49f5-ad46-ea8831fd7440-kube-api-access-mql82\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914243 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.914461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.914363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.915419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.915289 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/76f110e1-a005-49f5-ad46-ea8831fd7440-metrics-client-ca\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.917499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.917426 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.917977 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.917895 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.918076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.918018 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.918141 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.918090 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-grpc-tls\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.918547 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.918508 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.919633 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.919610 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/76f110e1-a005-49f5-ad46-ea8831fd7440-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:31.922775 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:31.922750 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mql82\" (UniqueName: \"kubernetes.io/projected/76f110e1-a005-49f5-ad46-ea8831fd7440-kube-api-access-mql82\") pod \"thanos-querier-668c9bc467-7kczj\" (UID: \"76f110e1-a005-49f5-ad46-ea8831fd7440\") " pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:32.024785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.024744 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:32.181157 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.181051 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-668c9bc467-7kczj"] Jul 11 23:32:32.238077 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.238028 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-qxk48" Jul 11 23:32:32.316047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.315945 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"3482791d1fed3610b62aacb55450b8601747cc76d48180b786485f95403b4b82"} Jul 11 23:32:32.319788 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.319753 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" event={"ID":"02312fb7-9694-4563-b92f-f2584c72ef7e","Type":"ContainerStarted","Data":"7ea4a7775b2e164de003753a81709a9e0c6f004ff4fedbc17149bf9ef609bc4c"} Jul 11 23:32:32.320011 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.319826 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" event={"ID":"02312fb7-9694-4563-b92f-f2584c72ef7e","Type":"ContainerStarted","Data":"a635fc681b5a254b10a1f7bbcb97f8354c7791da594aefdf2886cbb2e75ce6a7"} Jul 11 23:32:32.320011 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.319860 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" event={"ID":"02312fb7-9694-4563-b92f-f2584c72ef7e","Type":"ContainerStarted","Data":"53b108f04419a88df71816a7f797d7e36f7df11f9c34c04231df41e7ed3e19df"} Jul 11 23:32:32.324559 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.324439 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b5584696-rfvcm" event={"ID":"9cb5d34a-12a8-476c-8673-625595bdb724","Type":"ContainerStarted","Data":"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a"} Jul 11 23:32:32.324559 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.324527 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b5584696-rfvcm" event={"ID":"9cb5d34a-12a8-476c-8673-625595bdb724","Type":"ContainerStarted","Data":"e4989abeadaece6a9e7a63e006d3d8a2d16ce4dee9ea191cb735e7e9b0d9ebd6"} Jul 11 23:32:32.327230 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.327197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qt64h" event={"ID":"a0fc3079-83c6-4d06-bed2-d73eb17109fa","Type":"ContainerStarted","Data":"f9e7ae60b59369b05481a9df42f7f5156db2519c562b4e6af59b1674f35995a0"} Jul 11 23:32:32.329560 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.329497 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" event={"ID":"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8","Type":"ContainerStarted","Data":"f0682c9b12925c76e5699e640fe400ed4fca5ed827548cf169ee23c100d4532c"} Jul 11 23:32:32.332182 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.331773 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gstql" event={"ID":"f5de7bc3-7a27-4033-9303-380b33e254f9","Type":"ContainerStarted","Data":"308c678231eb55c6882f39025678cc0100e0362a4406a2d7a626ba3c22a749ae"} Jul 11 23:32:32.346982 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.345540 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-58b5584696-rfvcm" podStartSLOduration=4.345518037 podStartE2EDuration="4.345518037s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:32.345066439 +0000 UTC m=+72.096813943" watchObservedRunningTime="2026-07-11 23:32:32.345518037 +0000 UTC m=+72.097265542" Jul 11 23:32:32.364482 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:32.364098 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gstql" podStartSLOduration=66.171507085 podStartE2EDuration="1m11.364072355s" totalImagesPullingTime="5.19256527s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-11 23:31:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:32.36253 +0000 UTC m=+72.114277521" watchObservedRunningTime="2026-07-11 23:32:32.364072355 +0000 UTC m=+72.115819863" Jul 11 23:32:32.389969 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:32.389922 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76f110e1_a005_49f5_ad46_ea8831fd7440.slice/crio-651cada2185f0cef0d7750408c7851a0b072861b767fd837f3f560de8a3995f3 WatchSource:0}: Error finding container 651cada2185f0cef0d7750408c7851a0b072861b767fd837f3f560de8a3995f3: Status 404 returned error can't find the container with id 651cada2185f0cef0d7750408c7851a0b072861b767fd837f3f560de8a3995f3 Jul 11 23:32:33.336940 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:33.336908 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"651cada2185f0cef0d7750408c7851a0b072861b767fd837f3f560de8a3995f3"} Jul 11 23:32:33.338911 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:33.338881 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" event={"ID":"02312fb7-9694-4563-b92f-f2584c72ef7e","Type":"ContainerStarted","Data":"aae45907dca0ace9e9132e973640dfcf1072775b215b0846356b018e40e3f524"} Jul 11 23:32:33.356553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:33.356502 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-lc77j" podStartSLOduration=3.330755758 podStartE2EDuration="4.356485833s" totalImagesPullingTime="1.025730075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:33.354546421 +0000 UTC m=+73.106293923" watchObservedRunningTime="2026-07-11 23:32:33.356485833 +0000 UTC m=+73.108233330" Jul 11 23:32:34.137306 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.137262 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6798bd576f-4dg96"] Jul 11 23:32:34.142287 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.141341 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.145007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.144805 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-m6vq2t8dsqan\"" Jul 11 23:32:34.145007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.144809 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 11 23:32:34.145007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.144888 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 11 23:32:34.145007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.144929 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 11 23:32:34.145007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.144950 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-j24fw\"" Jul 11 23:32:34.145337 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.145301 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 11 23:32:34.150366 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.150311 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6798bd576f-4dg96"] Jul 11 23:32:34.236519 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-tls\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236528 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236575 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xg7rc\" (UniqueName: \"kubernetes.io/projected/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-kube-api-access-xg7rc\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236618 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-client-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236647 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-audit-log\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236682 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-client-certs\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.236839 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.236707 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-metrics-server-audit-profiles\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.337866 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.337821 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xg7rc\" (UniqueName: \"kubernetes.io/projected/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-kube-api-access-xg7rc\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.337866 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.337866 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-client-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338035 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-audit-log\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-client-certs\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-metrics-server-audit-profiles\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338176 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-tls\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338351 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338213 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338492 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-audit-log\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.338966 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.338942 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.339209 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.339187 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-metrics-server-audit-profiles\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.340797 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.340774 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-tls\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.340912 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.340774 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-client-ca-bundle\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.340912 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.340829 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-secret-metrics-server-client-certs\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.346581 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.346556 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xg7rc\" (UniqueName: \"kubernetes.io/projected/1374e254-a8cc-47cc-99dd-414a8bb6b7ff-kube-api-access-xg7rc\") pod \"metrics-server-6798bd576f-4dg96\" (UID: \"1374e254-a8cc-47cc-99dd-414a8bb6b7ff\") " pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.347052 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.347027 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06" exitCode=0 Jul 11 23:32:34.347591 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.347549 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06"} Jul 11 23:32:34.349456 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.349432 2580 generic.go:356] "Generic (PLEG): container finished" podID="a0fc3079-83c6-4d06-bed2-d73eb17109fa" containerID="5faa94c623dd7b2dbd05af52e7db114b4687e8b92e336c9c26b86111c3182e91" exitCode=0 Jul 11 23:32:34.350154 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.349567 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qt64h" event={"ID":"a0fc3079-83c6-4d06-bed2-d73eb17109fa","Type":"ContainerDied","Data":"5faa94c623dd7b2dbd05af52e7db114b4687e8b92e336c9c26b86111c3182e91"} Jul 11 23:32:34.353233 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.353198 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" event={"ID":"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8","Type":"ContainerStarted","Data":"82123afa07f11f15aa5a5ab01b46ccc0f16ccab9e80addbb7f7f0deaca351e4f"} Jul 11 23:32:34.353358 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.353255 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" event={"ID":"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8","Type":"ContainerStarted","Data":"600201547e4ad33a0ff9e368be762dedbf83809232d015924dea56adb4e041ed"} Jul 11 23:32:34.353358 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.353284 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" event={"ID":"e008a97f-3dfd-4ca0-b4fb-14717a72f1b8","Type":"ContainerStarted","Data":"946ca5807548b57889114f8190d43abc099f2e67ee793423939349c51b399a6e"} Jul 11 23:32:34.408489 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.406663 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-785778ff74-wlqh4" podStartSLOduration=3.584783797 podStartE2EDuration="5.406642507s" totalImagesPullingTime="1.82185871s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:34.405360718 +0000 UTC m=+74.157108221" watchObservedRunningTime="2026-07-11 23:32:34.406642507 +0000 UTC m=+74.158390011" Jul 11 23:32:34.453917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.453877 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:34.514246 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.514209 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j"] Jul 11 23:32:34.517997 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.517971 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.521357 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.521334 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-mzpmd\"" Jul 11 23:32:34.521537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.521406 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 11 23:32:34.524904 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.524835 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j"] Jul 11 23:32:34.605184 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.605150 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6798bd576f-4dg96"] Jul 11 23:32:34.640948 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.640904 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.641136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.640965 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b8nbq\" (UniqueName: \"kubernetes.io/projected/248746d9-aed0-4297-81ad-c44dc684a255-kube-api-access-b8nbq\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.742107 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.742066 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.742307 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.742122 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b8nbq\" (UniqueName: \"kubernetes.io/projected/248746d9-aed0-4297-81ad-c44dc684a255-kube-api-access-b8nbq\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.742307 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:34.742263 2580 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 11 23:32:34.742412 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:32:34.742337 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert podName:248746d9-aed0-4297-81ad-c44dc684a255 nodeName:}" failed. No retries permitted until 2026-07-11 23:32:35.24231453 +0000 UTC m=+74.994062014 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-mzr2j" (UID: "248746d9-aed0-4297-81ad-c44dc684a255") : secret "monitoring-plugin-cert" not found Jul 11 23:32:34.750433 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.750405 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b8nbq\" (UniqueName: \"kubernetes.io/projected/248746d9-aed0-4297-81ad-c44dc684a255-kube-api-access-b8nbq\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:34.979525 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.979494 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr"] Jul 11 23:32:34.984203 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.984181 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:34.986876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.986695 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-dhhzt\"" Jul 11 23:32:34.986876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.986744 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 11 23:32:34.986876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.986757 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 11 23:32:34.986876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.986759 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 11 23:32:34.986876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.986767 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 11 23:32:34.987220 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.987082 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 11 23:32:34.991359 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.991335 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 11 23:32:34.995395 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:34.995371 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr"] Jul 11 23:32:35.045921 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.045878 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-serving-certs-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.045921 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.045924 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pnwcv\" (UniqueName: \"kubernetes.io/projected/7cbaa007-2345-40a1-8734-1e5cf935534d-kube-api-access-pnwcv\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046180 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.045998 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-federate-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046180 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.046065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046180 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.046111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046180 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.046136 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-metrics-client-ca\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046344 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.046243 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.046344 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.046285 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.106502 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:35.106450 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1374e254_a8cc_47cc_99dd_414a8bb6b7ff.slice/crio-626f84b333ce71b2139aba0479ce6c289dbd44ff16c143454af31d372a2b08d2 WatchSource:0}: Error finding container 626f84b333ce71b2139aba0479ce6c289dbd44ff16c143454af31d372a2b08d2: Status 404 returned error can't find the container with id 626f84b333ce71b2139aba0479ce6c289dbd44ff16c143454af31d372a2b08d2 Jul 11 23:32:35.147101 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-serving-certs-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147292 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147107 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pnwcv\" (UniqueName: \"kubernetes.io/projected/7cbaa007-2345-40a1-8734-1e5cf935534d-kube-api-access-pnwcv\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147292 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147136 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-federate-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147292 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147184 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147292 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147236 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147292 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-metrics-client-ca\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147590 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147314 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147590 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.147954 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.147919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-serving-certs-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.148361 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.148334 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.148639 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.148616 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7cbaa007-2345-40a1-8734-1e5cf935534d-metrics-client-ca\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.149817 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.149794 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.150076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.150050 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-federate-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.150164 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.150087 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-telemeter-client-tls\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.150357 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.150336 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7cbaa007-2345-40a1-8734-1e5cf935534d-secret-telemeter-client\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.155211 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.155156 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pnwcv\" (UniqueName: \"kubernetes.io/projected/7cbaa007-2345-40a1-8734-1e5cf935534d-kube-api-access-pnwcv\") pod \"telemeter-client-7fcbdffc7-v4kdr\" (UID: \"7cbaa007-2345-40a1-8734-1e5cf935534d\") " pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.247831 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.247804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:35.250092 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.250062 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/248746d9-aed0-4297-81ad-c44dc684a255-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-mzr2j\" (UID: \"248746d9-aed0-4297-81ad-c44dc684a255\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:35.297174 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.297151 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" Jul 11 23:32:35.366063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.365921 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qt64h" event={"ID":"a0fc3079-83c6-4d06-bed2-d73eb17109fa","Type":"ContainerStarted","Data":"d995354915bf7d15951a37d8b0d0ec73f7724b5fca98d2e6a33ad54dd36d833a"} Jul 11 23:32:35.366063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.365984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-qt64h" event={"ID":"a0fc3079-83c6-4d06-bed2-d73eb17109fa","Type":"ContainerStarted","Data":"6fb35b0b873c685af634288713279b76786bc1c949b9fe2b9b981b19592ba236"} Jul 11 23:32:35.376600 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.376551 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"14965dc4619bc32972c3eedab2f101c11484b0c956d4a1e3de44ab6a8fe50a3f"} Jul 11 23:32:35.376750 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.376646 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"db73ca50bc4c12fbf59de83e43875272071a1e3459ff0e3892cd4259f20d8a40"} Jul 11 23:32:35.376750 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.376695 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"e0eeb172f2bea672b30fca0e54ea1f3660e145d5a94db63b052f9f38c3150e30"} Jul 11 23:32:35.378594 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.378559 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" event={"ID":"1374e254-a8cc-47cc-99dd-414a8bb6b7ff","Type":"ContainerStarted","Data":"626f84b333ce71b2139aba0479ce6c289dbd44ff16c143454af31d372a2b08d2"} Jul 11 23:32:35.387084 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.386934 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-qt64h" podStartSLOduration=4.567625307 podStartE2EDuration="6.386911903s" totalImagesPullingTime="1.739676789s" totalInitContainerRuntime="79.609807ms" isStatefulPod=true podCreationTimestamp="2026-07-11 23:32:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:35.384840655 +0000 UTC m=+75.136588185" watchObservedRunningTime="2026-07-11 23:32:35.386911903 +0000 UTC m=+75.138659406" Jul 11 23:32:35.439278 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.439245 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:35.457843 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.457757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr"] Jul 11 23:32:35.461871 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:35.461837 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cbaa007_2345_40a1_8734_1e5cf935534d.slice/crio-f0b961ebd3617ffcb60998f4ef051269d2d72e9058f4452383107e2ab4d95fdb WatchSource:0}: Error finding container f0b961ebd3617ffcb60998f4ef051269d2d72e9058f4452383107e2ab4d95fdb: Status 404 returned error can't find the container with id f0b961ebd3617ffcb60998f4ef051269d2d72e9058f4452383107e2ab4d95fdb Jul 11 23:32:35.587858 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:35.587815 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j"] Jul 11 23:32:35.591978 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:35.591949 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod248746d9_aed0_4297_81ad_c44dc684a255.slice/crio-66fec2a151c677c01ef1ac4adba1888731932528dce0f7972c132976d1771b11 WatchSource:0}: Error finding container 66fec2a151c677c01ef1ac4adba1888731932528dce0f7972c132976d1771b11: Status 404 returned error can't find the container with id 66fec2a151c677c01ef1ac4adba1888731932528dce0f7972c132976d1771b11 Jul 11 23:32:36.387599 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:36.387556 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" event={"ID":"7cbaa007-2345-40a1-8734-1e5cf935534d","Type":"ContainerStarted","Data":"f0b961ebd3617ffcb60998f4ef051269d2d72e9058f4452383107e2ab4d95fdb"} Jul 11 23:32:36.389860 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:36.389816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" event={"ID":"248746d9-aed0-4297-81ad-c44dc684a255","Type":"ContainerStarted","Data":"66fec2a151c677c01ef1ac4adba1888731932528dce0f7972c132976d1771b11"} Jul 11 23:32:38.408798 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.408743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" event={"ID":"1374e254-a8cc-47cc-99dd-414a8bb6b7ff","Type":"ContainerStarted","Data":"6617f7ec4da8eed58ea3fe425201aae6228ef92b3524f2f331db172a612e303e"} Jul 11 23:32:38.412184 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.412157 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51"} Jul 11 23:32:38.412291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.412232 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41"} Jul 11 23:32:38.412352 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.412307 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712"} Jul 11 23:32:38.412352 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.412349 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84"} Jul 11 23:32:38.414562 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.414535 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" event={"ID":"7cbaa007-2345-40a1-8734-1e5cf935534d","Type":"ContainerStarted","Data":"bd77ad533242fa9cba58fbdb401b7d5c22bd5e33cea91a3e90db33d63e8d2c70"} Jul 11 23:32:38.414655 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.414593 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" event={"ID":"7cbaa007-2345-40a1-8734-1e5cf935534d","Type":"ContainerStarted","Data":"cbdbe9937180ab8f8c2e9851b0faa9205cd2621a3e3124c7aa6f787e6b0f4d98"} Jul 11 23:32:38.414655 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.414622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" event={"ID":"7cbaa007-2345-40a1-8734-1e5cf935534d","Type":"ContainerStarted","Data":"96d5481c8df7cdecfe7473c83b01a65e0e13488d2af8becd76a3f2dbb5d575d0"} Jul 11 23:32:38.416356 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.416312 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" event={"ID":"248746d9-aed0-4297-81ad-c44dc684a255","Type":"ContainerStarted","Data":"b3ce78bdd5f79ced794e9c5b5869760e00e4a8fc0b11b6c32e0eb3aa02b98880"} Jul 11 23:32:38.416543 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.416525 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:38.420291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.419968 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"73bf0308b241daeac34e1d2ac4c8799df3389c1fb3a756bcdef3531808433560"} Jul 11 23:32:38.420291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.420044 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"b54654b3a9026efecc9b0481e0dc89e77caee1c95dbbfbb1a2bfc314fb19a7c6"} Jul 11 23:32:38.420291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.420091 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" event={"ID":"76f110e1-a005-49f5-ad46-ea8831fd7440","Type":"ContainerStarted","Data":"6a91cae1f8c70e936fb78408dd30f7b433c0d415fc0c114cd16c8ffb3f9582f8"} Jul 11 23:32:38.420291 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.420158 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:38.422623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.422602 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" Jul 11 23:32:38.425386 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.425349 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" podStartSLOduration=1.5123027690000002 podStartE2EDuration="4.425336979s" totalImagesPullingTime="2.91303421s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:38.423913659 +0000 UTC m=+78.175661163" watchObservedRunningTime="2026-07-11 23:32:38.425336979 +0000 UTC m=+78.177084487" Jul 11 23:32:38.442662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.442599 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7fcbdffc7-v4kdr" podStartSLOduration=1.885700474 podStartE2EDuration="4.442578168s" totalImagesPullingTime="2.556877694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:38.441204537 +0000 UTC m=+78.192952066" watchObservedRunningTime="2026-07-11 23:32:38.442578168 +0000 UTC m=+78.194325673" Jul 11 23:32:38.463437 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.463357 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" podStartSLOduration=2.025315768 podStartE2EDuration="7.463334682s" totalImagesPullingTime="5.438018914s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:31 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:38.461997629 +0000 UTC m=+78.213745132" watchObservedRunningTime="2026-07-11 23:32:38.463334682 +0000 UTC m=+78.215082187" Jul 11 23:32:38.478419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.478391 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:38.478715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.478668 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-mzr2j" podStartSLOduration=1.9977788159999998 podStartE2EDuration="4.478651845s" totalImagesPullingTime="2.480873029s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:38.476663157 +0000 UTC m=+78.228410675" watchObservedRunningTime="2026-07-11 23:32:38.478651845 +0000 UTC m=+78.230399349" Jul 11 23:32:38.478826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.478762 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:38.483776 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:38.483753 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:39.222880 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.222844 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:32:39.249595 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.249561 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:32:39.252920 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.252898 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.261277 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.261069 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:32:39.384588 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384550 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384600 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384675 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384717 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384736 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.384763 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.384761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v7kw\" (UniqueName: \"kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.433061 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.433020 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132"} Jul 11 23:32:39.433548 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.433131 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerStarted","Data":"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa"} Jul 11 23:32:39.437533 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.437506 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:32:39.457070 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.457014 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=4.271378704 podStartE2EDuration="9.456992641s" totalImagesPullingTime="5.163943797s" totalInitContainerRuntime="21.67014ms" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:30 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:39.455191578 +0000 UTC m=+79.206939136" watchObservedRunningTime="2026-07-11 23:32:39.456992641 +0000 UTC m=+79.208740145" Jul 11 23:32:39.485502 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485379 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485502 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485424 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485502 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485452 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6v7kw\" (UniqueName: \"kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485769 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485548 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485769 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485594 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485769 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485613 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.485769 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.485633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.486132 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.486109 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.486218 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.486203 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.486662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.486635 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.486937 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.486919 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.488529 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.488463 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.488662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.488507 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.493731 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.493706 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v7kw\" (UniqueName: \"kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw\") pod \"console-94949f487-2c8f4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.565701 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.565666 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:39.697092 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:39.697059 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:32:39.700391 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:32:39.700359 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod04979ed3_3afd_4a60_a435_3acbc698a0d4.slice/crio-430dbd0b6bf206c5486f890412278c2fe30ceff4f8314b79ab25dc7a9added04 WatchSource:0}: Error finding container 430dbd0b6bf206c5486f890412278c2fe30ceff4f8314b79ab25dc7a9added04: Status 404 returned error can't find the container with id 430dbd0b6bf206c5486f890412278c2fe30ceff4f8314b79ab25dc7a9added04 Jul 11 23:32:40.438274 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:40.438236 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94949f487-2c8f4" event={"ID":"04979ed3-3afd-4a60-a435-3acbc698a0d4","Type":"ContainerStarted","Data":"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa"} Jul 11 23:32:40.438752 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:40.438288 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94949f487-2c8f4" event={"ID":"04979ed3-3afd-4a60-a435-3acbc698a0d4","Type":"ContainerStarted","Data":"430dbd0b6bf206c5486f890412278c2fe30ceff4f8314b79ab25dc7a9added04"} Jul 11 23:32:40.455573 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:40.455518 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-94949f487-2c8f4" podStartSLOduration=1.455499997 podStartE2EDuration="1.455499997s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:32:39 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:32:40.453348559 +0000 UTC m=+80.205096085" watchObservedRunningTime="2026-07-11 23:32:40.455499997 +0000 UTC m=+80.207247500" Jul 11 23:32:44.439347 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:44.439313 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-668c9bc467-7kczj" Jul 11 23:32:49.565798 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:49.565760 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:49.566183 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:49.565822 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:49.570546 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:49.570522 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:50.483175 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:50.483147 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:32:50.524850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:50.524820 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:32:54.454313 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:54.454265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:32:54.454792 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:32:54.454337 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:33:02.334924 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:02.334893 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-b465v" Jul 11 23:33:05.459454 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:05.459389 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-58b5584696-rfvcm" podUID="9cb5d34a-12a8-476c-8673-625595bdb724" containerName="console" containerID="cri-o://5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a" gracePeriod=15 Jul 11 23:33:06.234484 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.234448 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:33:06.332653 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332616 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332653 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332657 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332723 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332742 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dm6d4\" (UniqueName: \"kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332764 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332784 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.332892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.332832 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config\") pod \"9cb5d34a-12a8-476c-8673-625595bdb724\" (UID: \"9cb5d34a-12a8-476c-8673-625595bdb724\") " Jul 11 23:33:06.333141 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.333115 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config" (OuterVolumeSpecName: "console-config") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:06.333211 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.333174 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:06.333272 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.333242 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca" (OuterVolumeSpecName: "service-ca") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:06.333370 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.333342 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:06.335213 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.335188 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:06.335213 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.335197 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4" (OuterVolumeSpecName: "kube-api-access-dm6d4") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "kube-api-access-dm6d4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:33:06.335346 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.335259 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9cb5d34a-12a8-476c-8673-625595bdb724" (UID: "9cb5d34a-12a8-476c-8673-625595bdb724"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:06.433882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433839 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-console-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.433882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433872 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-oauth-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.433882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433882 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.433882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433890 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dm6d4\" (UniqueName: \"kubernetes.io/projected/9cb5d34a-12a8-476c-8673-625595bdb724-kube-api-access-dm6d4\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.433882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433899 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9cb5d34a-12a8-476c-8673-625595bdb724-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.434176 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433907 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.434176 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.433915 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9cb5d34a-12a8-476c-8673-625595bdb724-console-oauth-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:06.550055 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.550024 2580 generic.go:356] "Generic (PLEG): container finished" podID="9cb5d34a-12a8-476c-8673-625595bdb724" containerID="5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a" exitCode=0 Jul 11 23:33:06.550450 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.550082 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-58b5584696-rfvcm" Jul 11 23:33:06.550450 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.550098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b5584696-rfvcm" event={"ID":"9cb5d34a-12a8-476c-8673-625595bdb724","Type":"ContainerDied","Data":"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a"} Jul 11 23:33:06.550450 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.550179 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-58b5584696-rfvcm" event={"ID":"9cb5d34a-12a8-476c-8673-625595bdb724","Type":"ContainerDied","Data":"e4989abeadaece6a9e7a63e006d3d8a2d16ce4dee9ea191cb735e7e9b0d9ebd6"} Jul 11 23:33:06.550450 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.550205 2580 scope.go:117] "RemoveContainer" containerID="5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a" Jul 11 23:33:06.559876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.559858 2580 scope.go:117] "RemoveContainer" containerID="5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a" Jul 11 23:33:06.560176 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:06.560147 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a\": container with ID starting with 5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a not found: ID does not exist" containerID="5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a" Jul 11 23:33:06.560234 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.560186 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a"} err="failed to get container status \"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a\": rpc error: code = NotFound desc = could not find container \"5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a\": container with ID starting with 5c75c5e2b1a8beb19d7eb9e805d517a52ca5d4eac2fe097461732bd03ff9039a not found: ID does not exist" Jul 11 23:33:06.571033 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.571006 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:33:06.574881 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.574858 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-58b5584696-rfvcm"] Jul 11 23:33:06.787125 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:06.787093 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9cb5d34a-12a8-476c-8673-625595bdb724" path="/var/lib/kubelet/pods/9cb5d34a-12a8-476c-8673-625595bdb724/volumes" Jul 11 23:33:11.323636 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:11.323607 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ngdqs_9529b946-8c65-47e9-90a3-1f33d9188cf1/serve-healthcheck-canary/0.log" Jul 11 23:33:14.459744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:14.459716 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:33:14.463555 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:14.463529 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6798bd576f-4dg96" Jul 11 23:33:15.545594 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.545532 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-664494f996-kgqwr" podUID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" containerName="console" containerID="cri-o://6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d" gracePeriod=15 Jul 11 23:33:15.790853 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.790829 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923672 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lsdvt\" (UniqueName: \"kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923743 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923798 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923834 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923867 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.923896 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config\") pod \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\" (UID: \"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5\") " Jul 11 23:33:15.928058 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.924677 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config" (OuterVolumeSpecName: "console-config") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:15.928529 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.928244 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca" (OuterVolumeSpecName: "service-ca") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:15.929091 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.929064 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:15.929394 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.929366 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt" (OuterVolumeSpecName: "kube-api-access-lsdvt") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "kube-api-access-lsdvt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:33:15.930076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.930047 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:15.930177 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:15.930129 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" (UID: "7d7f7c30-6c08-4610-8f2a-04cd4b8975d5"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:16.024757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024714 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.024757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024749 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lsdvt\" (UniqueName: \"kubernetes.io/projected/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-kube-api-access-lsdvt\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.024757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024764 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.024997 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024775 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-oauth-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.024997 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024787 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.024997 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.024800 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5-console-oauth-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:16.595512 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.595455 2580 generic.go:356] "Generic (PLEG): container finished" podID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" containerID="6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d" exitCode=0 Jul 11 23:33:16.595512 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.595499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664494f996-kgqwr" event={"ID":"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5","Type":"ContainerDied","Data":"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d"} Jul 11 23:33:16.595970 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.595542 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-664494f996-kgqwr" Jul 11 23:33:16.595970 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.595583 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-664494f996-kgqwr" event={"ID":"7d7f7c30-6c08-4610-8f2a-04cd4b8975d5","Type":"ContainerDied","Data":"365b20b9cae7ef385a6187382b269cc2d71ef7815b30e59d5e70d0757158a23a"} Jul 11 23:33:16.595970 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.595605 2580 scope.go:117] "RemoveContainer" containerID="6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d" Jul 11 23:33:16.603847 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.603832 2580 scope.go:117] "RemoveContainer" containerID="6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d" Jul 11 23:33:16.604079 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:16.604061 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d\": container with ID starting with 6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d not found: ID does not exist" containerID="6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d" Jul 11 23:33:16.604124 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.604088 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d"} err="failed to get container status \"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d\": rpc error: code = NotFound desc = could not find container \"6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d\": container with ID starting with 6ebb46ef970adde5594081af56df413aead2f575d76f2fd41e1878fefe428f3d not found: ID does not exist" Jul 11 23:33:16.614365 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.614344 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:33:16.617511 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.617490 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-664494f996-kgqwr"] Jul 11 23:33:16.787066 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:16.787036 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" path="/var/lib/kubelet/pods/7d7f7c30-6c08-4610-8f2a-04cd4b8975d5/volumes" Jul 11 23:33:50.045602 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.045562 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.045956 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="alertmanager" containerID="cri-o://995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" gracePeriod=120 Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.046038 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-web" containerID="cri-o://43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" gracePeriod=120 Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.046045 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-metric" containerID="cri-o://5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" gracePeriod=120 Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.046080 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="prom-label-proxy" containerID="cri-o://01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" gracePeriod=120 Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.046091 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="config-reloader" containerID="cri-o://c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" gracePeriod=120 Jul 11 23:33:50.046136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.046101 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy" containerID="cri-o://feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" gracePeriod=120 Jul 11 23:33:50.741662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741622 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" exitCode=0 Jul 11 23:33:50.741662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741646 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" exitCode=0 Jul 11 23:33:50.741662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741655 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" exitCode=0 Jul 11 23:33:50.741662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741663 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" exitCode=0 Jul 11 23:33:50.741979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741680 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132"} Jul 11 23:33:50.741979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741762 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51"} Jul 11 23:33:50.741979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741793 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712"} Jul 11 23:33:50.741979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:50.741825 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84"} Jul 11 23:33:51.286448 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.286422 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.325461 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325431 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.325647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325487 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.325647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325518 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.325647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325555 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.325647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325599 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.325647 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325631 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5c6ss\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325654 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325686 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325712 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325753 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325782 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325830 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325870 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric\") pod \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\" (UID: \"476b9d8d-5e3b-4492-b83b-eda3c11f3091\") " Jul 11 23:33:51.326076 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.325934 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:33:51.326499 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.326138 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-main-db\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.328221 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.328187 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:51.329664 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.329609 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.329664 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.329648 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:33:51.331637 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.331611 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume" (OuterVolumeSpecName: "config-volume") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.331775 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.331740 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out" (OuterVolumeSpecName: "config-out") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:33:51.331876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.331812 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.331876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.331815 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.332548 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.332522 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.332713 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.332690 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:33:51.332868 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.332842 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss" (OuterVolumeSpecName: "kube-api-access-5c6ss") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "kube-api-access-5c6ss". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:33:51.336303 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.336172 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.342901 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.342879 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config" (OuterVolumeSpecName: "web-config") pod "476b9d8d-5e3b-4492-b83b-eda3c11f3091" (UID: "476b9d8d-5e3b-4492-b83b-eda3c11f3091"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427017 2580 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-out\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427048 2580 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427059 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5c6ss\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-kube-api-access-5c6ss\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427068 2580 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-cluster-tls-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427077 2580 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/476b9d8d-5e3b-4492-b83b-eda3c11f3091-metrics-client-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427085 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427094 2580 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-config-volume\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427102 2580 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/476b9d8d-5e3b-4492-b83b-eda3c11f3091-tls-assets\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427110 2580 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-web-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427109 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427117 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427552 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427127 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.427552 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.427136 2580 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/476b9d8d-5e3b-4492-b83b-eda3c11f3091-secret-alertmanager-main-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:33:51.748589 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748552 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" exitCode=0 Jul 11 23:33:51.748589 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748578 2580 generic.go:356] "Generic (PLEG): container finished" podID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerID="43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" exitCode=0 Jul 11 23:33:51.748863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748639 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.748863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748637 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa"} Jul 11 23:33:51.748863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748767 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41"} Jul 11 23:33:51.748863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748809 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"476b9d8d-5e3b-4492-b83b-eda3c11f3091","Type":"ContainerDied","Data":"3482791d1fed3610b62aacb55450b8601747cc76d48180b786485f95403b4b82"} Jul 11 23:33:51.748863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.748837 2580 scope.go:117] "RemoveContainer" containerID="01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" Jul 11 23:33:51.757853 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.757834 2580 scope.go:117] "RemoveContainer" containerID="5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" Jul 11 23:33:51.765011 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.764985 2580 scope.go:117] "RemoveContainer" containerID="feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" Jul 11 23:33:51.771210 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.771189 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:51.772585 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.772571 2580 scope.go:117] "RemoveContainer" containerID="43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" Jul 11 23:33:51.778800 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.778778 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:51.779824 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.779812 2580 scope.go:117] "RemoveContainer" containerID="c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" Jul 11 23:33:51.786688 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.786668 2580 scope.go:117] "RemoveContainer" containerID="995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" Jul 11 23:33:51.794025 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.794008 2580 scope.go:117] "RemoveContainer" containerID="8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06" Jul 11 23:33:51.801164 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.801146 2580 scope.go:117] "RemoveContainer" containerID="01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" Jul 11 23:33:51.801428 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.801408 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132\": container with ID starting with 01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132 not found: ID does not exist" containerID="01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" Jul 11 23:33:51.801506 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.801437 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132"} err="failed to get container status \"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132\": rpc error: code = NotFound desc = could not find container \"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132\": container with ID starting with 01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132 not found: ID does not exist" Jul 11 23:33:51.801506 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.801459 2580 scope.go:117] "RemoveContainer" containerID="5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" Jul 11 23:33:51.801734 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.801716 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa\": container with ID starting with 5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa not found: ID does not exist" containerID="5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" Jul 11 23:33:51.801776 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.801740 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa"} err="failed to get container status \"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa\": rpc error: code = NotFound desc = could not find container \"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa\": container with ID starting with 5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa not found: ID does not exist" Jul 11 23:33:51.801776 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.801757 2580 scope.go:117] "RemoveContainer" containerID="feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" Jul 11 23:33:51.802003 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.801986 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51\": container with ID starting with feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51 not found: ID does not exist" containerID="feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" Jul 11 23:33:51.802043 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802008 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51"} err="failed to get container status \"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51\": rpc error: code = NotFound desc = could not find container \"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51\": container with ID starting with feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51 not found: ID does not exist" Jul 11 23:33:51.802043 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802022 2580 scope.go:117] "RemoveContainer" containerID="43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" Jul 11 23:33:51.802273 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.802255 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41\": container with ID starting with 43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41 not found: ID does not exist" containerID="43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" Jul 11 23:33:51.802312 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802279 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41"} err="failed to get container status \"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41\": rpc error: code = NotFound desc = could not find container \"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41\": container with ID starting with 43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41 not found: ID does not exist" Jul 11 23:33:51.802312 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802293 2580 scope.go:117] "RemoveContainer" containerID="c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" Jul 11 23:33:51.802568 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.802552 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712\": container with ID starting with c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712 not found: ID does not exist" containerID="c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" Jul 11 23:33:51.802612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802571 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712"} err="failed to get container status \"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712\": rpc error: code = NotFound desc = could not find container \"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712\": container with ID starting with c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712 not found: ID does not exist" Jul 11 23:33:51.802612 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802583 2580 scope.go:117] "RemoveContainer" containerID="995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" Jul 11 23:33:51.802802 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.802786 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84\": container with ID starting with 995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84 not found: ID does not exist" containerID="995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" Jul 11 23:33:51.802838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802807 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84"} err="failed to get container status \"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84\": rpc error: code = NotFound desc = could not find container \"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84\": container with ID starting with 995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84 not found: ID does not exist" Jul 11 23:33:51.802838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.802821 2580 scope.go:117] "RemoveContainer" containerID="8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06" Jul 11 23:33:51.803084 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:33:51.803024 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06\": container with ID starting with 8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06 not found: ID does not exist" containerID="8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06" Jul 11 23:33:51.803084 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803043 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06"} err="failed to get container status \"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06\": rpc error: code = NotFound desc = could not find container \"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06\": container with ID starting with 8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06 not found: ID does not exist" Jul 11 23:33:51.803084 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803056 2580 scope.go:117] "RemoveContainer" containerID="01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132" Jul 11 23:33:51.803495 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803316 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132"} err="failed to get container status \"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132\": rpc error: code = NotFound desc = could not find container \"01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132\": container with ID starting with 01ec6bb5b23108242634f97139dd333de0cad86d66e493e586f1bbf796034132 not found: ID does not exist" Jul 11 23:33:51.803495 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803334 2580 scope.go:117] "RemoveContainer" containerID="5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa" Jul 11 23:33:51.803899 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803867 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa"} err="failed to get container status \"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa\": rpc error: code = NotFound desc = could not find container \"5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa\": container with ID starting with 5b576b5ffb4529640f8805381ad8f3c70bfa7ac470a9501fb330f8fbccbb5faa not found: ID does not exist" Jul 11 23:33:51.803899 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.803907 2580 scope.go:117] "RemoveContainer" containerID="feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51" Jul 11 23:33:51.804305 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.804269 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51"} err="failed to get container status \"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51\": rpc error: code = NotFound desc = could not find container \"feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51\": container with ID starting with feb18d114ed7f8851e340d65a4bba180947822657a5fa46f94c10441acbb8b51 not found: ID does not exist" Jul 11 23:33:51.804305 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.804306 2580 scope.go:117] "RemoveContainer" containerID="43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41" Jul 11 23:33:51.804935 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.804911 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41"} err="failed to get container status \"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41\": rpc error: code = NotFound desc = could not find container \"43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41\": container with ID starting with 43d06a9cf243a24cd5f67b5a7db6eb5694f59549be3f10ec0c22f173c8773d41 not found: ID does not exist" Jul 11 23:33:51.805045 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.804937 2580 scope.go:117] "RemoveContainer" containerID="c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712" Jul 11 23:33:51.805214 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805193 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712"} err="failed to get container status \"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712\": rpc error: code = NotFound desc = could not find container \"c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712\": container with ID starting with c9f4659deac83fa19d1df25a27f615715d2c493e128389dde72cb2b4ce018712 not found: ID does not exist" Jul 11 23:33:51.805287 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805215 2580 scope.go:117] "RemoveContainer" containerID="995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84" Jul 11 23:33:51.805950 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805513 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84"} err="failed to get container status \"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84\": rpc error: code = NotFound desc = could not find container \"995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84\": container with ID starting with 995412c91ee5459120e9828df1c1c22be19bff2949706118bc8e10f3ccd4bd84 not found: ID does not exist" Jul 11 23:33:51.805950 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805538 2580 scope.go:117] "RemoveContainer" containerID="8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06" Jul 11 23:33:51.805950 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805647 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:51.805950 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.805779 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06"} err="failed to get container status \"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06\": rpc error: code = NotFound desc = could not find container \"8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06\": container with ID starting with 8625d0907e6e5870ce2270b5132378f6ca88be7f990423f17f4306ab79231c06 not found: ID does not exist" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806038 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" containerName="console" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806052 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" containerName="console" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806063 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="config-reloader" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806072 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="config-reloader" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806088 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="prom-label-proxy" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806097 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="prom-label-proxy" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806109 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-web" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806118 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-web" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806139 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="init-config-reloader" Jul 11 23:33:51.806160 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806148 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="init-config-reloader" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806190 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806199 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806207 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-metric" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806215 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-metric" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806248 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9cb5d34a-12a8-476c-8673-625595bdb724" containerName="console" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806257 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="9cb5d34a-12a8-476c-8673-625595bdb724" containerName="console" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806267 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="alertmanager" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806275 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="alertmanager" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806354 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="alertmanager" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806364 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-metric" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806374 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="config-reloader" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806382 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="9cb5d34a-12a8-476c-8673-625595bdb724" containerName="console" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806390 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy-web" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806398 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="kube-rbac-proxy" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806405 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" containerName="prom-label-proxy" Jul 11 23:33:51.806537 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.806414 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7d7f7c30-6c08-4610-8f2a-04cd4b8975d5" containerName="console" Jul 11 23:33:51.809104 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.809089 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.811112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.811089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 11 23:33:51.811435 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.811419 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 11 23:33:51.811903 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.811885 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 11 23:33:51.811979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.811918 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 11 23:33:51.812047 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.812019 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 11 23:33:51.812132 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.812115 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-27w9l\"" Jul 11 23:33:51.812302 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.812286 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 11 23:33:51.812385 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.812287 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 11 23:33:51.812649 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.812634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 11 23:33:51.817436 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.817418 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 11 23:33:51.824520 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.824495 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:51.831070 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831183 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831087 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831183 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831120 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831510 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831510 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831435 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-out\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831798 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831911 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831812 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.831911 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.832037 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831943 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.832037 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831967 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mq2l2\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-kube-api-access-mq2l2\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.832037 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.831999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-web-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.832037 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.832023 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.832231 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.832046 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933188 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933148 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933188 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933186 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933205 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mq2l2\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-kube-api-access-mq2l2\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933228 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-web-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933243 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933419 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933265 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933656 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933410 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933788 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933876 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933812 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933931 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933990 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933937 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-out\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.933990 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.934081 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.933991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.934132 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.934077 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.934262 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.934229 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.934362 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.934344 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936533 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936400 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-tls-assets\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936533 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936442 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-web-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936533 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936525 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-volume\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936832 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936812 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/836cfc2e-0604-4065-bd43-6baf81cf90dd-config-out\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936873 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936860 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936958 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936939 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.936996 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.936951 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.937135 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.937120 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.938038 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.938021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/836cfc2e-0604-4065-bd43-6baf81cf90dd-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:51.940438 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:51.940417 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mq2l2\" (UniqueName: \"kubernetes.io/projected/836cfc2e-0604-4065-bd43-6baf81cf90dd-kube-api-access-mq2l2\") pod \"alertmanager-main-0\" (UID: \"836cfc2e-0604-4065-bd43-6baf81cf90dd\") " pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:52.118934 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.118836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 11 23:33:52.255724 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.255682 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 11 23:33:52.257860 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:33:52.257831 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod836cfc2e_0604_4065_bd43_6baf81cf90dd.slice/crio-e7078c7f597d932bd7e4c37ae8648a8ad0ea1dcddc8ebfe3c5828b29c52f3a3f WatchSource:0}: Error finding container e7078c7f597d932bd7e4c37ae8648a8ad0ea1dcddc8ebfe3c5828b29c52f3a3f: Status 404 returned error can't find the container with id e7078c7f597d932bd7e4c37ae8648a8ad0ea1dcddc8ebfe3c5828b29c52f3a3f Jul 11 23:33:52.758788 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.758748 2580 generic.go:356] "Generic (PLEG): container finished" podID="836cfc2e-0604-4065-bd43-6baf81cf90dd" containerID="ec86e930513a634f193fa0d711bd12056f8d32169099b3a932547c25d1c6d47f" exitCode=0 Jul 11 23:33:52.759173 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.758807 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerDied","Data":"ec86e930513a634f193fa0d711bd12056f8d32169099b3a932547c25d1c6d47f"} Jul 11 23:33:52.759173 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.758993 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"e7078c7f597d932bd7e4c37ae8648a8ad0ea1dcddc8ebfe3c5828b29c52f3a3f"} Jul 11 23:33:52.787623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:52.787597 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="476b9d8d-5e3b-4492-b83b-eda3c11f3091" path="/var/lib/kubelet/pods/476b9d8d-5e3b-4492-b83b-eda3c11f3091/volumes" Jul 11 23:33:53.339405 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.339326 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:33:53.341331 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.341311 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.353915 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.353885 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:33:53.447995 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.447957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.447995 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.447999 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.448256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.448029 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.448256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.448122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.448256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.448165 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.448256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.448211 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kbttm\" (UniqueName: \"kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.448401 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.448256 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.548944 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.548917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549123 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.548962 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549123 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.548996 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549123 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549028 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kbttm\" (UniqueName: \"kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549259 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549328 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549311 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549384 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549730 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549870 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549762 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.549870 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.549841 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.550050 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.550031 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.551608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.551578 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.551708 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.551583 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.555842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.555821 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kbttm\" (UniqueName: \"kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm\") pod \"console-867c84bccf-tgp42\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.651187 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.651091 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:33:53.767963 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.767928 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"f4c3c8684367d981423531734a290b02ca50bc6a983e819a06032684feb1411c"} Jul 11 23:33:53.768338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.768008 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"cc3223f5e31810ab5f5c015aeb50ab3ce229c6084b800e4965ff1bb1fdd0fbcb"} Jul 11 23:33:53.768338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.768037 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"eae41e9629f4a570e52f3f514970058b9531ac73a335638b69dbc09b6c93db99"} Jul 11 23:33:53.768338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.768068 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"16292c34654213930c123dc3dd8b3bd90fd0fbed841aea96619525f5f2b3c8d2"} Jul 11 23:33:53.768338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.768095 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"5922216ec10aaf4f4b14d40788b7990c3d6925d40639fc087d2e09661c921450"} Jul 11 23:33:53.768338 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.768120 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"836cfc2e-0604-4065-bd43-6baf81cf90dd","Type":"ContainerStarted","Data":"606f3f5d3c5a401d49c3bb09d49e5947008dd002f7e4eba6ef5e9262e8ceed64"} Jul 11 23:33:53.780574 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.780545 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:33:53.783388 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:33:53.783364 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbfe83135_662f_44ac_bcc4_3766fce5ecf3.slice/crio-1aca1b65a953a01f286f0fd69d0ecea097f7cfd7f6ed0193049271d3fc5ce35f WatchSource:0}: Error finding container 1aca1b65a953a01f286f0fd69d0ecea097f7cfd7f6ed0193049271d3fc5ce35f: Status 404 returned error can't find the container with id 1aca1b65a953a01f286f0fd69d0ecea097f7cfd7f6ed0193049271d3fc5ce35f Jul 11 23:33:53.791883 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:53.791845 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.773356813 podStartE2EDuration="2.791831045s" totalImagesPullingTime="0s" totalInitContainerRuntime="18.474232ms" isStatefulPod=false podCreationTimestamp="2026-07-11 23:33:51 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:33:53.790590584 +0000 UTC m=+153.542338086" watchObservedRunningTime="2026-07-11 23:33:53.791831045 +0000 UTC m=+153.543578548" Jul 11 23:33:54.773231 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:54.773194 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867c84bccf-tgp42" event={"ID":"bfe83135-662f-44ac-bcc4-3766fce5ecf3","Type":"ContainerStarted","Data":"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649"} Jul 11 23:33:54.773654 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:54.773278 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867c84bccf-tgp42" event={"ID":"bfe83135-662f-44ac-bcc4-3766fce5ecf3","Type":"ContainerStarted","Data":"1aca1b65a953a01f286f0fd69d0ecea097f7cfd7f6ed0193049271d3fc5ce35f"} Jul 11 23:33:54.789855 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:33:54.789809 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-867c84bccf-tgp42" podStartSLOduration=1.7897944319999999 podStartE2EDuration="1.789794432s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:33:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:33:54.787847929 +0000 UTC m=+154.539595434" watchObservedRunningTime="2026-07-11 23:33:54.789794432 +0000 UTC m=+154.541541934" Jul 11 23:34:03.652263 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:03.652212 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:34:03.652263 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:03.652275 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:34:03.656969 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:03.656942 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:34:03.813343 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:03.813312 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:34:03.847818 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:03.847784 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:34:28.872991 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:28.872952 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-94949f487-2c8f4" podUID="04979ed3-3afd-4a60-a435-3acbc698a0d4" containerName="console" containerID="cri-o://25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa" gracePeriod=15 Jul 11 23:34:29.114918 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.114894 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:34:29.164422 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164333 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164422 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164381 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164669 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164504 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164669 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164533 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164669 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164563 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164669 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164587 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6v7kw\" (UniqueName: \"kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164669 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164610 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca\") pod \"04979ed3-3afd-4a60-a435-3acbc698a0d4\" (UID: \"04979ed3-3afd-4a60-a435-3acbc698a0d4\") " Jul 11 23:34:29.164912 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164798 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:34:29.164912 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.164872 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-oauth-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.165053 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.165025 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca" (OuterVolumeSpecName: "service-ca") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:34:29.165053 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.165033 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:34:29.165211 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.165182 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config" (OuterVolumeSpecName: "console-config") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:34:29.166772 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.166741 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:34:29.166890 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.166768 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw" (OuterVolumeSpecName: "kube-api-access-6v7kw") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "kube-api-access-6v7kw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:34:29.166890 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.166790 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "04979ed3-3afd-4a60-a435-3acbc698a0d4" (UID: "04979ed3-3afd-4a60-a435-3acbc698a0d4"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:34:29.265214 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265171 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.265214 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265206 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.265214 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265219 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.265455 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265231 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6v7kw\" (UniqueName: \"kubernetes.io/projected/04979ed3-3afd-4a60-a435-3acbc698a0d4-kube-api-access-6v7kw\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.265455 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265243 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/04979ed3-3afd-4a60-a435-3acbc698a0d4-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.265455 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.265254 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/04979ed3-3afd-4a60-a435-3acbc698a0d4-console-oauth-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:29.913978 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.913939 2580 generic.go:356] "Generic (PLEG): container finished" podID="04979ed3-3afd-4a60-a435-3acbc698a0d4" containerID="25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa" exitCode=0 Jul 11 23:34:29.914463 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.913989 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-94949f487-2c8f4" Jul 11 23:34:29.914463 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.914022 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94949f487-2c8f4" event={"ID":"04979ed3-3afd-4a60-a435-3acbc698a0d4","Type":"ContainerDied","Data":"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa"} Jul 11 23:34:29.914463 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.914072 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-94949f487-2c8f4" event={"ID":"04979ed3-3afd-4a60-a435-3acbc698a0d4","Type":"ContainerDied","Data":"430dbd0b6bf206c5486f890412278c2fe30ceff4f8314b79ab25dc7a9added04"} Jul 11 23:34:29.914463 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.914092 2580 scope.go:117] "RemoveContainer" containerID="25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa" Jul 11 23:34:29.922628 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.922610 2580 scope.go:117] "RemoveContainer" containerID="25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa" Jul 11 23:34:29.922879 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:34:29.922862 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa\": container with ID starting with 25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa not found: ID does not exist" containerID="25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa" Jul 11 23:34:29.922925 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.922890 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa"} err="failed to get container status \"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa\": rpc error: code = NotFound desc = could not find container \"25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa\": container with ID starting with 25bba017efcbc8bb40a0e0652a47ee52944e2d9f98629a5964b10084bebdddfa not found: ID does not exist" Jul 11 23:34:29.934072 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.934046 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:34:29.935300 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:29.935282 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-94949f487-2c8f4"] Jul 11 23:34:30.787448 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:30.787419 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="04979ed3-3afd-4a60-a435-3acbc698a0d4" path="/var/lib/kubelet/pods/04979ed3-3afd-4a60-a435-3acbc698a0d4/volumes" Jul 11 23:34:35.583095 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.583057 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg"] Jul 11 23:34:35.583500 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.583383 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="04979ed3-3afd-4a60-a435-3acbc698a0d4" containerName="console" Jul 11 23:34:35.583500 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.583393 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="04979ed3-3afd-4a60-a435-3acbc698a0d4" containerName="console" Jul 11 23:34:35.583500 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.583447 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="04979ed3-3afd-4a60-a435-3acbc698a0d4" containerName="console" Jul 11 23:34:35.586332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.586312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.588384 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.588363 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ggj6m\"" Jul 11 23:34:35.588692 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.588675 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 23:34:35.589286 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.589270 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 23:34:35.595017 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.594992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg"] Jul 11 23:34:35.611184 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.611155 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.611364 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.611189 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk4w7\" (UniqueName: \"kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.611364 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.611213 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.711930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.711897 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.711930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.711932 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zk4w7\" (UniqueName: \"kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.712165 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.711954 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.712301 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.712281 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.712336 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.712312 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.719060 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.719036 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk4w7\" (UniqueName: \"kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:35.895904 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:35.895819 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:36.024485 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:36.024440 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg"] Jul 11 23:34:36.027160 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:34:36.027135 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb336de86_7e23_4f29_a46a_14b9b21d166f.slice/crio-87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92 WatchSource:0}: Error finding container 87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92: Status 404 returned error can't find the container with id 87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92 Jul 11 23:34:36.950107 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:36.950070 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" event={"ID":"b336de86-7e23-4f29-a46a-14b9b21d166f","Type":"ContainerStarted","Data":"87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92"} Jul 11 23:34:41.972967 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:41.972929 2580 generic.go:356] "Generic (PLEG): container finished" podID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerID="a30b0a66c0cdbcd8236cc718862b50215f060f575682607a3a962ffc99d6c157" exitCode=0 Jul 11 23:34:41.973429 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:41.973013 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" event={"ID":"b336de86-7e23-4f29-a46a-14b9b21d166f","Type":"ContainerDied","Data":"a30b0a66c0cdbcd8236cc718862b50215f060f575682607a3a962ffc99d6c157"} Jul 11 23:34:44.986705 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:44.986666 2580 generic.go:356] "Generic (PLEG): container finished" podID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerID="aa21b4f5938713172cde3dc63d84c342a69520fe217cb1705dd7f7ee12659818" exitCode=0 Jul 11 23:34:44.987190 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:44.986728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" event={"ID":"b336de86-7e23-4f29-a46a-14b9b21d166f","Type":"ContainerDied","Data":"aa21b4f5938713172cde3dc63d84c342a69520fe217cb1705dd7f7ee12659818"} Jul 11 23:34:52.019991 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:52.019950 2580 generic.go:356] "Generic (PLEG): container finished" podID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerID="3f6a0bd3f045d32dcee03950b736ec023d225841a4d4d7b8568b11a487b412a7" exitCode=0 Jul 11 23:34:52.020429 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:52.020014 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" event={"ID":"b336de86-7e23-4f29-a46a-14b9b21d166f","Type":"ContainerDied","Data":"3f6a0bd3f045d32dcee03950b736ec023d225841a4d4d7b8568b11a487b412a7"} Jul 11 23:34:53.151449 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.151419 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:53.273066 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.273015 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util\") pod \"b336de86-7e23-4f29-a46a-14b9b21d166f\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " Jul 11 23:34:53.273267 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.273089 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zk4w7\" (UniqueName: \"kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7\") pod \"b336de86-7e23-4f29-a46a-14b9b21d166f\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " Jul 11 23:34:53.273267 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.273144 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle\") pod \"b336de86-7e23-4f29-a46a-14b9b21d166f\" (UID: \"b336de86-7e23-4f29-a46a-14b9b21d166f\") " Jul 11 23:34:53.273835 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.273809 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle" (OuterVolumeSpecName: "bundle") pod "b336de86-7e23-4f29-a46a-14b9b21d166f" (UID: "b336de86-7e23-4f29-a46a-14b9b21d166f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:34:53.275379 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.275330 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7" (OuterVolumeSpecName: "kube-api-access-zk4w7") pod "b336de86-7e23-4f29-a46a-14b9b21d166f" (UID: "b336de86-7e23-4f29-a46a-14b9b21d166f"). InnerVolumeSpecName "kube-api-access-zk4w7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:34:53.280256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.280219 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util" (OuterVolumeSpecName: "util") pod "b336de86-7e23-4f29-a46a-14b9b21d166f" (UID: "b336de86-7e23-4f29-a46a-14b9b21d166f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:34:53.373943 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.373885 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:53.373943 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.373931 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/b336de86-7e23-4f29-a46a-14b9b21d166f-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:53.373943 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:53.373942 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zk4w7\" (UniqueName: \"kubernetes.io/projected/b336de86-7e23-4f29-a46a-14b9b21d166f-kube-api-access-zk4w7\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:34:54.031105 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:54.031067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" event={"ID":"b336de86-7e23-4f29-a46a-14b9b21d166f","Type":"ContainerDied","Data":"87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92"} Jul 11 23:34:54.031272 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:54.031119 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="87d34a57c632f3e8fc895d905b7906c883fff64cc94dee2649736df43720ae92" Jul 11 23:34:54.031317 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:54.031271 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540mm8pg" Jul 11 23:34:58.445173 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445142 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r"] Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445460 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="pull" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445483 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="pull" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445492 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="util" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445497 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="util" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445511 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="extract" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445517 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="extract" Jul 11 23:34:58.445663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.445564 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b336de86-7e23-4f29-a46a-14b9b21d166f" containerName="extract" Jul 11 23:34:58.492223 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.492184 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r"] Jul 11 23:34:58.492387 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.492306 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.495035 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.495009 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 11 23:34:58.495175 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.495159 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-d9knv\"" Jul 11 23:34:58.495252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.495240 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 11 23:34:58.621530 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.621462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e1326e-bf0e-4111-9dde-b56467399302-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.621700 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.621552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctbmr\" (UniqueName: \"kubernetes.io/projected/a0e1326e-bf0e-4111-9dde-b56467399302-kube-api-access-ctbmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.722875 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.722797 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ctbmr\" (UniqueName: \"kubernetes.io/projected/a0e1326e-bf0e-4111-9dde-b56467399302-kube-api-access-ctbmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.723007 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.722898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e1326e-bf0e-4111-9dde-b56467399302-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.723247 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.723232 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/a0e1326e-bf0e-4111-9dde-b56467399302-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.730832 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.730805 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctbmr\" (UniqueName: \"kubernetes.io/projected/a0e1326e-bf0e-4111-9dde-b56467399302-kube-api-access-ctbmr\") pod \"cert-manager-operator-controller-manager-55c444bf86-lpl8r\" (UID: \"a0e1326e-bf0e-4111-9dde-b56467399302\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.802136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.802102 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" Jul 11 23:34:58.958506 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:58.958461 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r"] Jul 11 23:34:58.961241 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:34:58.961213 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0e1326e_bf0e_4111_9dde_b56467399302.slice/crio-33eda3dba9a3512a9984adb542d05c51bead35bfeb9377bef0d267e56469f14b WatchSource:0}: Error finding container 33eda3dba9a3512a9984adb542d05c51bead35bfeb9377bef0d267e56469f14b: Status 404 returned error can't find the container with id 33eda3dba9a3512a9984adb542d05c51bead35bfeb9377bef0d267e56469f14b Jul 11 23:34:59.052181 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:34:59.052085 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" event={"ID":"a0e1326e-bf0e-4111-9dde-b56467399302","Type":"ContainerStarted","Data":"33eda3dba9a3512a9984adb542d05c51bead35bfeb9377bef0d267e56469f14b"} Jul 11 23:35:01.063542 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:01.063504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" event={"ID":"a0e1326e-bf0e-4111-9dde-b56467399302","Type":"ContainerStarted","Data":"2c255246217e8cba4e8b39de81704ab69abf1989404e1ba41f922b37558153e2"} Jul 11 23:35:01.082522 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:01.082449 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-lpl8r" podStartSLOduration=1.379412105 podStartE2EDuration="3.082429906s" totalImagesPullingTime="1.703017801s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:34:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:01.0806986 +0000 UTC m=+220.832446128" watchObservedRunningTime="2026-07-11 23:35:01.082429906 +0000 UTC m=+220.834177439" Jul 11 23:35:02.180379 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.180342 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl"] Jul 11 23:35:02.182746 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.182729 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.185340 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.185308 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 23:35:02.185522 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.185502 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ggj6m\"" Jul 11 23:35:02.185644 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.185627 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 23:35:02.195732 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.195702 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl"] Jul 11 23:35:02.253554 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.253519 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.253714 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.253602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdbc7\" (UniqueName: \"kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.253714 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.253627 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.354271 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.354222 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jdbc7\" (UniqueName: \"kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.354271 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.354273 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.354525 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.354399 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.354655 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.354638 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.354758 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.354740 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.362080 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.362049 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdbc7\" (UniqueName: \"kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.497414 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.497377 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:02.630582 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:02.630528 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl"] Jul 11 23:35:02.634777 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:02.634740 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc183fafe_44a6_4b10_9956_237d25fbd86c.slice/crio-8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2 WatchSource:0}: Error finding container 8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2: Status 404 returned error can't find the container with id 8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2 Jul 11 23:35:03.073877 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:03.073846 2580 generic.go:356] "Generic (PLEG): container finished" podID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerID="d4bac669f05a766ebd4669dc40beb34605b1a22af5f175ca03c567b9602c4038" exitCode=0 Jul 11 23:35:03.074063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:03.073938 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" event={"ID":"c183fafe-44a6-4b10-9956-237d25fbd86c","Type":"ContainerDied","Data":"d4bac669f05a766ebd4669dc40beb34605b1a22af5f175ca03c567b9602c4038"} Jul 11 23:35:03.074063 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:03.073988 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" event={"ID":"c183fafe-44a6-4b10-9956-237d25fbd86c","Type":"ContainerStarted","Data":"8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2"} Jul 11 23:35:05.389561 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.389529 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-np827"] Jul 11 23:35:05.391674 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.391658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.393878 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.393852 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 11 23:35:05.394739 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.394719 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 11 23:35:05.394846 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.394724 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-ld5jf\"" Jul 11 23:35:05.399810 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.399791 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-np827"] Jul 11 23:35:05.480826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.480794 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.480986 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.480853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-khz9d\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-kube-api-access-khz9d\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.581457 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.581406 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.581675 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.581498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-khz9d\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-kube-api-access-khz9d\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.588884 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.588852 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.589013 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.588968 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-khz9d\" (UniqueName: \"kubernetes.io/projected/97565fc9-89d8-470d-8b56-bbb448a7edf2-kube-api-access-khz9d\") pod \"cert-manager-cainjector-559cdd8c87-np827\" (UID: \"97565fc9-89d8-470d-8b56-bbb448a7edf2\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.709025 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.708983 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" Jul 11 23:35:05.841656 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:05.841611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-np827"] Jul 11 23:35:05.843737 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:05.843712 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97565fc9_89d8_470d_8b56_bbb448a7edf2.slice/crio-c6fa0b12da8081b5dca6fbac923b700535780194b341d0ee4cda0fa131aef3ad WatchSource:0}: Error finding container c6fa0b12da8081b5dca6fbac923b700535780194b341d0ee4cda0fa131aef3ad: Status 404 returned error can't find the container with id c6fa0b12da8081b5dca6fbac923b700535780194b341d0ee4cda0fa131aef3ad Jul 11 23:35:06.091256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:06.091163 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" event={"ID":"97565fc9-89d8-470d-8b56-bbb448a7edf2","Type":"ContainerStarted","Data":"c6fa0b12da8081b5dca6fbac923b700535780194b341d0ee4cda0fa131aef3ad"} Jul 11 23:35:06.093156 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:06.093129 2580 generic.go:356] "Generic (PLEG): container finished" podID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerID="f05e48ab5d696a6c7d4f43146e444692af0188ade3bcc5d5ac3935b5a06ec484" exitCode=0 Jul 11 23:35:06.093269 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:06.093178 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" event={"ID":"c183fafe-44a6-4b10-9956-237d25fbd86c","Type":"ContainerDied","Data":"f05e48ab5d696a6c7d4f43146e444692af0188ade3bcc5d5ac3935b5a06ec484"} Jul 11 23:35:07.101634 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:07.101598 2580 generic.go:356] "Generic (PLEG): container finished" podID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerID="39a6a21c0a075e8e503e48a7df2ab59a4fc2368409ddb463534fee31005d9b22" exitCode=0 Jul 11 23:35:07.102093 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:07.101687 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" event={"ID":"c183fafe-44a6-4b10-9956-237d25fbd86c","Type":"ContainerDied","Data":"39a6a21c0a075e8e503e48a7df2ab59a4fc2368409ddb463534fee31005d9b22"} Jul 11 23:35:08.507341 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.507315 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:08.608582 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.608554 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jdbc7\" (UniqueName: \"kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7\") pod \"c183fafe-44a6-4b10-9956-237d25fbd86c\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " Jul 11 23:35:08.608728 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.608615 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle\") pod \"c183fafe-44a6-4b10-9956-237d25fbd86c\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " Jul 11 23:35:08.608728 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.608660 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util\") pod \"c183fafe-44a6-4b10-9956-237d25fbd86c\" (UID: \"c183fafe-44a6-4b10-9956-237d25fbd86c\") " Jul 11 23:35:08.609121 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.609085 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle" (OuterVolumeSpecName: "bundle") pod "c183fafe-44a6-4b10-9956-237d25fbd86c" (UID: "c183fafe-44a6-4b10-9956-237d25fbd86c"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:35:08.610719 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.610684 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7" (OuterVolumeSpecName: "kube-api-access-jdbc7") pod "c183fafe-44a6-4b10-9956-237d25fbd86c" (UID: "c183fafe-44a6-4b10-9956-237d25fbd86c"). InnerVolumeSpecName "kube-api-access-jdbc7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:35:08.612781 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.612744 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util" (OuterVolumeSpecName: "util") pod "c183fafe-44a6-4b10-9956-237d25fbd86c" (UID: "c183fafe-44a6-4b10-9956-237d25fbd86c"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:35:08.709386 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.709343 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jdbc7\" (UniqueName: \"kubernetes.io/projected/c183fafe-44a6-4b10-9956-237d25fbd86c-kube-api-access-jdbc7\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:08.709386 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.709381 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:08.709386 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:08.709390 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c183fafe-44a6-4b10-9956-237d25fbd86c-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:09.112846 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:09.112760 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" Jul 11 23:35:09.112846 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:09.112791 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fmwfhl" event={"ID":"c183fafe-44a6-4b10-9956-237d25fbd86c","Type":"ContainerDied","Data":"8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2"} Jul 11 23:35:09.112846 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:09.112844 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="8671a24bbd079cb210d953f7fc3d82004fed57610953abcab07e05368d6f66b2" Jul 11 23:35:09.114318 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:09.114297 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" event={"ID":"97565fc9-89d8-470d-8b56-bbb448a7edf2","Type":"ContainerStarted","Data":"3100ad211cac1728067777cfbb0402556b21e0d259af5956405540d3ce94061b"} Jul 11 23:35:09.129186 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:09.129134 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-np827" podStartSLOduration=1.430604201 podStartE2EDuration="4.129118977s" totalImagesPullingTime="2.698514776s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:09.128418734 +0000 UTC m=+228.880166239" watchObservedRunningTime="2026-07-11 23:35:09.129118977 +0000 UTC m=+228.880866479" Jul 11 23:35:10.006667 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006633 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gcbt5"] Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006955 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="pull" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006966 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="pull" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006975 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="util" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006980 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="util" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006992 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="extract" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.006997 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="extract" Jul 11 23:35:10.007064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.007059 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c183fafe-44a6-4b10-9956-237d25fbd86c" containerName="extract" Jul 11 23:35:10.009987 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.009965 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.012140 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.012115 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-zpdgh\"" Jul 11 23:35:10.019637 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.019615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gcbt5"] Jul 11 23:35:10.119629 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.119601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.119629 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.119628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt6jh\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-kube-api-access-kt6jh\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.220100 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.220059 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.220100 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.220101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kt6jh\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-kube-api-access-kt6jh\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.227564 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.227534 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-bound-sa-token\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.227709 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.227693 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt6jh\" (UniqueName: \"kubernetes.io/projected/d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0-kube-api-access-kt6jh\") pod \"cert-manager-webhook-6486ffd997-gcbt5\" (UID: \"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0\") " pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.319801 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.319703 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:10.451512 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:10.451480 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-6486ffd997-gcbt5"] Jul 11 23:35:10.454503 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:10.454455 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd36ac4d9_fe35_4ca8_8e31_2dae90e8b0f0.slice/crio-dc0e051c43b48a612bd8deab0c9dfd1613fb47abee5aca1a45db7f3436f8542c WatchSource:0}: Error finding container dc0e051c43b48a612bd8deab0c9dfd1613fb47abee5aca1a45db7f3436f8542c: Status 404 returned error can't find the container with id dc0e051c43b48a612bd8deab0c9dfd1613fb47abee5aca1a45db7f3436f8542c Jul 11 23:35:11.129623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:11.129588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" event={"ID":"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0","Type":"ContainerStarted","Data":"f524db466b47238d0993695624a9fa147cd8d8efd3dda8a51c8e12ba34eeea35"} Jul 11 23:35:11.129623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:11.129630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" event={"ID":"d36ac4d9-fe35-4ca8-8e31-2dae90e8b0f0","Type":"ContainerStarted","Data":"dc0e051c43b48a612bd8deab0c9dfd1613fb47abee5aca1a45db7f3436f8542c"} Jul 11 23:35:11.130020 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:11.129708 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:11.146161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:11.146105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" podStartSLOduration=2.1460890790000002 podStartE2EDuration="2.146089079s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:09 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:11.144069953 +0000 UTC m=+230.895817490" watchObservedRunningTime="2026-07-11 23:35:11.146089079 +0000 UTC m=+230.897836581" Jul 11 23:35:12.148239 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.148194 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cjtf8"] Jul 11 23:35:12.151629 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.151604 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.153969 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.153938 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-5jdv2\"" Jul 11 23:35:12.159879 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.159856 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cjtf8"] Jul 11 23:35:12.240429 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.240395 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-bound-sa-token\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.240622 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.240557 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnksp\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-kube-api-access-vnksp\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.341538 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.341501 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-bound-sa-token\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.341698 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.341587 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vnksp\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-kube-api-access-vnksp\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.348886 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.348861 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-bound-sa-token\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.349018 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.348989 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnksp\" (UniqueName: \"kubernetes.io/projected/21393013-4629-43a9-a582-38626ec3f182-kube-api-access-vnksp\") pod \"cert-manager-55b4d954d5-cjtf8\" (UID: \"21393013-4629-43a9-a582-38626ec3f182\") " pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.462060 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.461967 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-cjtf8" Jul 11 23:35:12.589428 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:12.589364 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-cjtf8"] Jul 11 23:35:12.591602 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:12.591575 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod21393013_4629_43a9_a582_38626ec3f182.slice/crio-8be2dc5d0dcbffb1c3b5ec36b49abf59b41e0ac087f49740e49fe4380375225d WatchSource:0}: Error finding container 8be2dc5d0dcbffb1c3b5ec36b49abf59b41e0ac087f49740e49fe4380375225d: Status 404 returned error can't find the container with id 8be2dc5d0dcbffb1c3b5ec36b49abf59b41e0ac087f49740e49fe4380375225d Jul 11 23:35:13.140849 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:13.140816 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cjtf8" event={"ID":"21393013-4629-43a9-a582-38626ec3f182","Type":"ContainerStarted","Data":"119170cc34e153d779799217129f30bc29eef5c945cdd7765ef878d3d7176eb6"} Jul 11 23:35:13.140849 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:13.140855 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-cjtf8" event={"ID":"21393013-4629-43a9-a582-38626ec3f182","Type":"ContainerStarted","Data":"8be2dc5d0dcbffb1c3b5ec36b49abf59b41e0ac087f49740e49fe4380375225d"} Jul 11 23:35:13.155747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:13.155701 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-cjtf8" podStartSLOduration=1.155685098 podStartE2EDuration="1.155685098s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:12 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:13.154886375 +0000 UTC m=+232.906633898" watchObservedRunningTime="2026-07-11 23:35:13.155685098 +0000 UTC m=+232.907432602" Jul 11 23:35:15.780694 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.780657 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k"] Jul 11 23:35:15.784039 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.784022 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.786553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.786534 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 11 23:35:15.786748 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.786735 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 11 23:35:15.787385 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.787367 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-8rrl8\"" Jul 11 23:35:15.792706 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.792680 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k"] Jul 11 23:35:15.873382 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.873342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bm7wv\" (UniqueName: \"kubernetes.io/projected/5eca7cd1-b772-4025-a4b9-8b01fb68a490-kube-api-access-bm7wv\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.873382 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.873390 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eca7cd1-b772-4025-a4b9-8b01fb68a490-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.974233 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.974195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bm7wv\" (UniqueName: \"kubernetes.io/projected/5eca7cd1-b772-4025-a4b9-8b01fb68a490-kube-api-access-bm7wv\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.974233 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.974233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eca7cd1-b772-4025-a4b9-8b01fb68a490-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.974626 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.974609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/5eca7cd1-b772-4025-a4b9-8b01fb68a490-tmp\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:15.981528 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:15.981496 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bm7wv\" (UniqueName: \"kubernetes.io/projected/5eca7cd1-b772-4025-a4b9-8b01fb68a490-kube-api-access-bm7wv\") pod \"openshift-lws-operator-fd8ccff4c-sx72k\" (UID: \"5eca7cd1-b772-4025-a4b9-8b01fb68a490\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:16.099457 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:16.099364 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" Jul 11 23:35:16.247999 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:16.247966 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k"] Jul 11 23:35:16.250294 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:16.250267 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5eca7cd1_b772_4025_a4b9_8b01fb68a490.slice/crio-bd70381eb3dd92af4fe17ab628deb078d2043d41262494c2952f9df6ee4d8a57 WatchSource:0}: Error finding container bd70381eb3dd92af4fe17ab628deb078d2043d41262494c2952f9df6ee4d8a57: Status 404 returned error can't find the container with id bd70381eb3dd92af4fe17ab628deb078d2043d41262494c2952f9df6ee4d8a57 Jul 11 23:35:17.137295 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:17.137259 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-6486ffd997-gcbt5" Jul 11 23:35:17.186186 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:17.186144 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" event={"ID":"5eca7cd1-b772-4025-a4b9-8b01fb68a490","Type":"ContainerStarted","Data":"bd70381eb3dd92af4fe17ab628deb078d2043d41262494c2952f9df6ee4d8a57"} Jul 11 23:35:18.192592 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:18.192558 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" event={"ID":"5eca7cd1-b772-4025-a4b9-8b01fb68a490","Type":"ContainerStarted","Data":"6d4f48c73b70f6eb0cff874ee20386b53a4cf5b069301c5893f5c132e46b3204"} Jul 11 23:35:18.207687 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:18.207632 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-sx72k" podStartSLOduration=1.425966473 podStartE2EDuration="3.207615176s" totalImagesPullingTime="1.781648703s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:18.20661077 +0000 UTC m=+237.958358275" watchObservedRunningTime="2026-07-11 23:35:18.207615176 +0000 UTC m=+237.959362678" Jul 11 23:35:20.426878 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.426838 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw"] Jul 11 23:35:20.430527 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.430502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.432792 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.432766 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 23:35:20.432930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.432821 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 23:35:20.433678 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.433659 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ggj6m\"" Jul 11 23:35:20.437383 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.437346 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw"] Jul 11 23:35:20.614196 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.614159 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxj2j\" (UniqueName: \"kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.614415 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.614249 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.614415 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.614342 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.714945 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.714857 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.714945 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.714912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.715124 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.714959 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxj2j\" (UniqueName: \"kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.715346 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.715323 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.715382 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.715330 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:20.725644 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.725623 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 11 23:35:20.735894 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.735874 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 11 23:35:20.745208 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:20.745183 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxj2j\" (UniqueName: \"kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:21.045014 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:21.044983 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-ggj6m\"" Jul 11 23:35:21.051332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:21.051312 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:21.180002 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:21.179980 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw"] Jul 11 23:35:21.182103 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:21.182074 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a5f134_f058_4f47_8fbe_31397b63ceba.slice/crio-984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1 WatchSource:0}: Error finding container 984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1: Status 404 returned error can't find the container with id 984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1 Jul 11 23:35:21.206732 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:21.206694 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" event={"ID":"c9a5f134-f058-4f47-8fbe-31397b63ceba","Type":"ContainerStarted","Data":"984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1"} Jul 11 23:35:21.309215 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:35:21.309152 2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a5f134_f058_4f47_8fbe_31397b63ceba.slice/crio-a92aa40cd7454af2df389239b5c39f3b94579f7439c7cbf36195fad12b48e152.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9a5f134_f058_4f47_8fbe_31397b63ceba.slice/crio-conmon-a92aa40cd7454af2df389239b5c39f3b94579f7439c7cbf36195fad12b48e152.scope\": RecentStats: unable to find data in memory cache]" Jul 11 23:35:22.212632 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:22.212546 2580 generic.go:356] "Generic (PLEG): container finished" podID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerID="a92aa40cd7454af2df389239b5c39f3b94579f7439c7cbf36195fad12b48e152" exitCode=0 Jul 11 23:35:22.212994 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:22.212622 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" event={"ID":"c9a5f134-f058-4f47-8fbe-31397b63ceba","Type":"ContainerDied","Data":"a92aa40cd7454af2df389239b5c39f3b94579f7439c7cbf36195fad12b48e152"} Jul 11 23:35:23.222602 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:23.222512 2580 generic.go:356] "Generic (PLEG): container finished" podID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerID="5c9767bc45f6937a9a67aa913fecfec27b4b79e221d12afaddd04bbd5cfe106d" exitCode=0 Jul 11 23:35:23.222943 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:23.222598 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" event={"ID":"c9a5f134-f058-4f47-8fbe-31397b63ceba","Type":"ContainerDied","Data":"5c9767bc45f6937a9a67aa913fecfec27b4b79e221d12afaddd04bbd5cfe106d"} Jul 11 23:35:24.229623 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:24.229588 2580 generic.go:356] "Generic (PLEG): container finished" podID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerID="dbae37aba65ac9fbc766b6e50699ccb0828a97df299d0e89739415a1cb5aa144" exitCode=0 Jul 11 23:35:24.230080 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:24.229683 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" event={"ID":"c9a5f134-f058-4f47-8fbe-31397b63ceba","Type":"ContainerDied","Data":"dbae37aba65ac9fbc766b6e50699ccb0828a97df299d0e89739415a1cb5aa144"} Jul 11 23:35:25.362816 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.362791 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:25.451700 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.451668 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mxj2j\" (UniqueName: \"kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j\") pod \"c9a5f134-f058-4f47-8fbe-31397b63ceba\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " Jul 11 23:35:25.451877 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.451731 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle\") pod \"c9a5f134-f058-4f47-8fbe-31397b63ceba\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " Jul 11 23:35:25.451877 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.451826 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util\") pod \"c9a5f134-f058-4f47-8fbe-31397b63ceba\" (UID: \"c9a5f134-f058-4f47-8fbe-31397b63ceba\") " Jul 11 23:35:25.452460 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.452435 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle" (OuterVolumeSpecName: "bundle") pod "c9a5f134-f058-4f47-8fbe-31397b63ceba" (UID: "c9a5f134-f058-4f47-8fbe-31397b63ceba"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:35:25.453813 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.453792 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j" (OuterVolumeSpecName: "kube-api-access-mxj2j") pod "c9a5f134-f058-4f47-8fbe-31397b63ceba" (UID: "c9a5f134-f058-4f47-8fbe-31397b63ceba"). InnerVolumeSpecName "kube-api-access-mxj2j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:35:25.457510 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.457464 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util" (OuterVolumeSpecName: "util") pod "c9a5f134-f058-4f47-8fbe-31397b63ceba" (UID: "c9a5f134-f058-4f47-8fbe-31397b63ceba"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:35:25.552403 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.552314 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mxj2j\" (UniqueName: \"kubernetes.io/projected/c9a5f134-f058-4f47-8fbe-31397b63ceba-kube-api-access-mxj2j\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:25.552403 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.552347 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:25.552403 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:25.552360 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c9a5f134-f058-4f47-8fbe-31397b63ceba-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:35:26.240761 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:26.240728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" event={"ID":"c9a5f134-f058-4f47-8fbe-31397b63ceba","Type":"ContainerDied","Data":"984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1"} Jul 11 23:35:26.240919 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:26.240774 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="984ba56dbf88c008c94f6628d1bce81d32f0ddf54a2bb2f9120a5ce5aba135d1" Jul 11 23:35:26.240919 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:26.240777 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afx5gnw" Jul 11 23:35:38.545480 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545446 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg"] Jul 11 23:35:38.545863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545815 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="pull" Jul 11 23:35:38.545863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545827 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="pull" Jul 11 23:35:38.545863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545843 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="util" Jul 11 23:35:38.545863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545849 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="util" Jul 11 23:35:38.545863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545857 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="extract" Jul 11 23:35:38.546082 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545866 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="extract" Jul 11 23:35:38.546082 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.545933 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c9a5f134-f058-4f47-8fbe-31397b63ceba" containerName="extract" Jul 11 23:35:38.551844 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.551824 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.554613 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.554588 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 11 23:35:38.554613 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.554598 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 11 23:35:38.554813 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.554654 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 11 23:35:38.554813 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.554671 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 11 23:35:38.554813 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.554802 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-dzfxt\"" Jul 11 23:35:38.561882 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.561858 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg"] Jul 11 23:35:38.669098 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.669055 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.669098 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.669105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sbcpd\" (UniqueName: \"kubernetes.io/projected/10e36855-88f4-491a-8b60-b9feacc5b23f-kube-api-access-sbcpd\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.669320 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.669139 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.669320 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.669161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.669320 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.669204 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770048 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770010 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770084 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sbcpd\" (UniqueName: \"kubernetes.io/projected/10e36855-88f4-491a-8b60-b9feacc5b23f-kube-api-access-sbcpd\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770152 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.770575 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.770543 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-manifests\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.772504 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.772465 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/10e36855-88f4-491a-8b60-b9feacc5b23f-tmp\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.772717 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.772699 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-apiservice-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.772758 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.772740 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/10e36855-88f4-491a-8b60-b9feacc5b23f-webhook-cert\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.777912 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.777888 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sbcpd\" (UniqueName: \"kubernetes.io/projected/10e36855-88f4-491a-8b60-b9feacc5b23f-kube-api-access-sbcpd\") pod \"opendatahub-operator-controller-manager-6d55578dd8-4jvrg\" (UID: \"10e36855-88f4-491a-8b60-b9feacc5b23f\") " pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:38.862764 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:38.862668 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:39.035702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:39.035678 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg"] Jul 11 23:35:39.038452 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:35:39.038425 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10e36855_88f4_491a_8b60_b9feacc5b23f.slice/crio-3df1ae79510e1ce78ab82e9a6676f44b3dda87d7dcfce1963cc8f39876861508 WatchSource:0}: Error finding container 3df1ae79510e1ce78ab82e9a6676f44b3dda87d7dcfce1963cc8f39876861508: Status 404 returned error can't find the container with id 3df1ae79510e1ce78ab82e9a6676f44b3dda87d7dcfce1963cc8f39876861508 Jul 11 23:35:39.306173 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:39.306137 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" event={"ID":"10e36855-88f4-491a-8b60-b9feacc5b23f","Type":"ContainerStarted","Data":"3df1ae79510e1ce78ab82e9a6676f44b3dda87d7dcfce1963cc8f39876861508"} Jul 11 23:35:42.324906 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:42.324865 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" event={"ID":"10e36855-88f4-491a-8b60-b9feacc5b23f","Type":"ContainerStarted","Data":"e311ee96eebeedd4d6bdce1c904ab56670e4df8367c43fe62e46a380c7b3374a"} Jul 11 23:35:43.331417 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:43.331379 2580 generic.go:356] "Generic (PLEG): container finished" podID="10e36855-88f4-491a-8b60-b9feacc5b23f" containerID="e311ee96eebeedd4d6bdce1c904ab56670e4df8367c43fe62e46a380c7b3374a" exitCode=0 Jul 11 23:35:43.331835 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:43.331428 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" event={"ID":"10e36855-88f4-491a-8b60-b9feacc5b23f","Type":"ContainerDied","Data":"e311ee96eebeedd4d6bdce1c904ab56670e4df8367c43fe62e46a380c7b3374a"} Jul 11 23:35:44.338642 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:44.338605 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" event={"ID":"10e36855-88f4-491a-8b60-b9feacc5b23f","Type":"ContainerStarted","Data":"36bfb05368b06bca366933f732438fd583c170c7b96feb4db65234f959ee63f7"} Jul 11 23:35:44.339037 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:44.338736 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:35:44.363791 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:44.363737 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" podStartSLOduration=2.044877094 podStartE2EDuration="6.36371537s" totalImagesPullingTime="3.223398714s" totalInitContainerRuntime="1.095439562s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:35:38 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:35:44.359767628 +0000 UTC m=+264.111515127" watchObservedRunningTime="2026-07-11 23:35:44.36371537 +0000 UTC m=+264.115462874" Jul 11 23:35:55.346313 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:35:55.346280 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-6d55578dd8-4jvrg" Jul 11 23:36:03.516891 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.516855 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf"] Jul 11 23:36:03.525672 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.525647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.527965 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.527936 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 11 23:36:03.528821 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.528793 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 11 23:36:03.528821 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.528816 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 11 23:36:03.529083 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.529052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-j98pt\"" Jul 11 23:36:03.529213 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.529124 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 11 23:36:03.535619 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.532620 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf"] Jul 11 23:36:03.692164 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.692122 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7357cf84-807d-42d6-8f25-c558a74dd56f-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.692164 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.692174 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7357cf84-807d-42d6-8f25-c558a74dd56f-tmp\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.692373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.692255 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8ftq\" (UniqueName: \"kubernetes.io/projected/7357cf84-807d-42d6-8f25-c558a74dd56f-kube-api-access-f8ftq\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.793381 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.793295 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7357cf84-807d-42d6-8f25-c558a74dd56f-tmp\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.793553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.793382 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f8ftq\" (UniqueName: \"kubernetes.io/projected/7357cf84-807d-42d6-8f25-c558a74dd56f-kube-api-access-f8ftq\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.793553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.793411 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7357cf84-807d-42d6-8f25-c558a74dd56f-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.795687 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.795662 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7357cf84-807d-42d6-8f25-c558a74dd56f-tmp\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.795927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.795908 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7357cf84-807d-42d6-8f25-c558a74dd56f-tls-certs\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.800775 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.800754 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8ftq\" (UniqueName: \"kubernetes.io/projected/7357cf84-807d-42d6-8f25-c558a74dd56f-kube-api-access-f8ftq\") pod \"kube-auth-proxy-7bd58cf797-nmwmf\" (UID: \"7357cf84-807d-42d6-8f25-c558a74dd56f\") " pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.842643 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.842601 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" Jul 11 23:36:03.978443 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:03.978104 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf"] Jul 11 23:36:04.435383 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:04.435348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" event={"ID":"7357cf84-807d-42d6-8f25-c558a74dd56f","Type":"ContainerStarted","Data":"f9fa23216b796eb93dbc3324421af762c11a4271c44fad74aeaec1f96f4c469d"} Jul 11 23:36:06.538301 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.538264 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl"] Jul 11 23:36:06.542038 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.542009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.545898 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.545732 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 11 23:36:06.545898 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.545807 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-tkxpb\"" Jul 11 23:36:06.545898 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.545879 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 11 23:36:06.545898 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.545816 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 11 23:36:06.556247 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.556222 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl"] Jul 11 23:36:06.721590 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.721547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-manager-config\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.721806 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.721607 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pkvvn\" (UniqueName: \"kubernetes.io/projected/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-kube-api-access-pkvvn\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.721806 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.721659 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.721806 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.721697 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-metrics-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.823069 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.822963 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-manager-config\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.823069 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.823009 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pkvvn\" (UniqueName: \"kubernetes.io/projected/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-kube-api-access-pkvvn\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.823069 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.823045 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.823360 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.823080 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-metrics-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.823774 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.823749 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-manager-config\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.825692 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.825673 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-metrics-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.825749 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.825723 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-cert\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.831004 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.830984 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pkvvn\" (UniqueName: \"kubernetes.io/projected/b47a2fef-49dd-45b8-80ec-2b6a175ae81c-kube-api-access-pkvvn\") pod \"lws-controller-manager-59d758c946-4ljpl\" (UID: \"b47a2fef-49dd-45b8-80ec-2b6a175ae81c\") " pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.855995 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.855960 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:06.999485 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:06.999438 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl"] Jul 11 23:36:07.002895 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:36:07.002864 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb47a2fef_49dd_45b8_80ec_2b6a175ae81c.slice/crio-4069ff921192e2d29593f3771495e831213c692647d37c452f9e8ae1cd638d1b WatchSource:0}: Error finding container 4069ff921192e2d29593f3771495e831213c692647d37c452f9e8ae1cd638d1b: Status 404 returned error can't find the container with id 4069ff921192e2d29593f3771495e831213c692647d37c452f9e8ae1cd638d1b Jul 11 23:36:07.456485 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:07.456431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" event={"ID":"7357cf84-807d-42d6-8f25-c558a74dd56f","Type":"ContainerStarted","Data":"abed9a9d138acf862836a137f238f48251373ff8fd7cb286e1db93e3ea767c8d"} Jul 11 23:36:07.457635 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:07.457609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" event={"ID":"b47a2fef-49dd-45b8-80ec-2b6a175ae81c","Type":"ContainerStarted","Data":"4069ff921192e2d29593f3771495e831213c692647d37c452f9e8ae1cd638d1b"} Jul 11 23:36:07.476721 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:07.476665 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7bd58cf797-nmwmf" podStartSLOduration=1.91246683 podStartE2EDuration="4.476649983s" totalImagesPullingTime="2.564183153s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:07.474162915 +0000 UTC m=+287.225910457" watchObservedRunningTime="2026-07-11 23:36:07.476649983 +0000 UTC m=+287.228397485" Jul 11 23:36:09.469052 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:09.469012 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" event={"ID":"b47a2fef-49dd-45b8-80ec-2b6a175ae81c","Type":"ContainerStarted","Data":"6e140764f80becdeedb0f7dab67e68e7d70dbf64be19718c0d3df8df82852019"} Jul 11 23:36:09.469578 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:09.469148 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:09.498895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:09.498838 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" podStartSLOduration=1.863712453 podStartE2EDuration="3.498821218s" totalImagesPullingTime="1.635108765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:36:09.497621237 +0000 UTC m=+289.249368788" watchObservedRunningTime="2026-07-11 23:36:09.498821218 +0000 UTC m=+289.250568721" Jul 11 23:36:20.475816 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:20.475783 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-59d758c946-4ljpl" Jul 11 23:36:20.680603 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:20.680567 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:36:20.684913 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:20.684888 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 11 23:36:20.685783 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:36:20.685756 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:37:14.533800 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.533722 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz"] Jul 11 23:37:14.537283 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.537263 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.539897 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.539874 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kk9jd\"" Jul 11 23:37:14.540062 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.540044 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 11 23:37:14.540124 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.540088 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 11 23:37:14.547064 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.547038 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz"] Jul 11 23:37:14.608067 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.608025 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qkxnk\" (UniqueName: \"kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.608266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.608079 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.608266 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.608141 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.709362 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.709325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qkxnk\" (UniqueName: \"kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.709362 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.709372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.709654 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.709404 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.709785 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.709769 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.709836 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.709808 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.716997 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.716972 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qkxnk\" (UniqueName: \"kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.848103 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.848012 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:14.980728 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:14.980695 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz"] Jul 11 23:37:14.984251 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:14.984225 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7093ffb8_9b32_45c0_ac31_9bef83bca313.slice/crio-7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9 WatchSource:0}: Error finding container 7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9: Status 404 returned error can't find the container with id 7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9 Jul 11 23:37:15.128692 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.128608 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd"] Jul 11 23:37:15.131967 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.131950 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.139503 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.139461 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd"] Jul 11 23:37:15.213601 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.213563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kwf9j\" (UniqueName: \"kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.213760 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.213608 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.213760 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.213651 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.314870 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.314830 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kwf9j\" (UniqueName: \"kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.314870 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.314876 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.315126 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.314915 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.315256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.315239 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.315331 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.315303 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.322010 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.321984 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kwf9j\" (UniqueName: \"kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.452899 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.452802 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:15.535223 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.535187 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c"] Jul 11 23:37:15.539688 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.539628 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.545723 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.545660 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c"] Jul 11 23:37:15.592681 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.592658 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd"] Jul 11 23:37:15.594445 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:15.594423 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3a1f993_3bb8_4655_831d_6621fd9a1078.slice/crio-1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37 WatchSource:0}: Error finding container 1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37: Status 404 returned error can't find the container with id 1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37 Jul 11 23:37:15.616630 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.616606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-74kff\" (UniqueName: \"kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.616740 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.616658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.616740 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.616687 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.717540 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.717428 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-74kff\" (UniqueName: \"kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.717540 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.717525 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.717731 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.717554 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.717918 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.717900 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.717953 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.717923 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.724535 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.724512 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-74kff\" (UniqueName: \"kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.789632 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.789596 2580 generic.go:356] "Generic (PLEG): container finished" podID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerID="4dfe8a508e9ce0ff23d04bef709dfad17d950f232c58fb763665fde90166aedc" exitCode=0 Jul 11 23:37:15.789826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.789666 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" event={"ID":"c3a1f993-3bb8-4655-831d-6621fd9a1078","Type":"ContainerDied","Data":"4dfe8a508e9ce0ff23d04bef709dfad17d950f232c58fb763665fde90166aedc"} Jul 11 23:37:15.789826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.789728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" event={"ID":"c3a1f993-3bb8-4655-831d-6621fd9a1078","Type":"ContainerStarted","Data":"1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37"} Jul 11 23:37:15.790330 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.790309 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 11 23:37:15.791220 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.791199 2580 generic.go:356] "Generic (PLEG): container finished" podID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerID="ef2840581e3adc86ee707ede700748eac1dfc51391bd66388c147c39e6be832c" exitCode=0 Jul 11 23:37:15.791318 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.791231 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" event={"ID":"7093ffb8-9b32-45c0-ac31-9bef83bca313","Type":"ContainerDied","Data":"ef2840581e3adc86ee707ede700748eac1dfc51391bd66388c147c39e6be832c"} Jul 11 23:37:15.791318 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.791278 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" event={"ID":"7093ffb8-9b32-45c0-ac31-9bef83bca313","Type":"ContainerStarted","Data":"7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9"} Jul 11 23:37:15.853052 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.853009 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:15.932132 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.929978 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr"] Jul 11 23:37:15.934668 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.934642 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:15.945632 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:15.945601 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr"] Jul 11 23:37:16.009807 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.009752 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c"] Jul 11 23:37:16.011919 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:16.011892 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10eaa2bc_ab69_49a9_a1e4_1baf05dfc8a0.slice/crio-e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc WatchSource:0}: Error finding container e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc: Status 404 returned error can't find the container with id e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc Jul 11 23:37:16.021566 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.021542 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfclr\" (UniqueName: \"kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.021659 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.021643 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.021702 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.021668 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.122838 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.122810 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.122982 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.122846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.122982 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.122889 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rfclr\" (UniqueName: \"kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.123193 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.123175 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.123262 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.123242 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.130399 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.130373 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfclr\" (UniqueName: \"kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.250296 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.250255 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:16.380996 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.380971 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr"] Jul 11 23:37:16.382740 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:16.382714 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4bdbd3ad_5c9b_42fa_8722_17f9a1f86dda.slice/crio-2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa WatchSource:0}: Error finding container 2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa: Status 404 returned error can't find the container with id 2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa Jul 11 23:37:16.798172 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.798140 2580 generic.go:356] "Generic (PLEG): container finished" podID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerID="2dd035ca0c58028af141c126acba4a020defefb404959da155847ea34b15b433" exitCode=0 Jul 11 23:37:16.798595 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.798182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" event={"ID":"7093ffb8-9b32-45c0-ac31-9bef83bca313","Type":"ContainerDied","Data":"2dd035ca0c58028af141c126acba4a020defefb404959da155847ea34b15b433"} Jul 11 23:37:16.799755 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.799739 2580 generic.go:356] "Generic (PLEG): container finished" podID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerID="2ac581a4b1a685866da3987feffd547f1a1b8057a073bd596b7afce7755a7b1f" exitCode=0 Jul 11 23:37:16.799863 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.799832 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" event={"ID":"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda","Type":"ContainerDied","Data":"2ac581a4b1a685866da3987feffd547f1a1b8057a073bd596b7afce7755a7b1f"} Jul 11 23:37:16.799979 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.799886 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" event={"ID":"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda","Type":"ContainerStarted","Data":"2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa"} Jul 11 23:37:16.801886 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.801862 2580 generic.go:356] "Generic (PLEG): container finished" podID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerID="3cb75690c137c99a5f0cc02d84626e577dc6fd424e07967662f345473656e3da" exitCode=0 Jul 11 23:37:16.802005 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.801887 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" event={"ID":"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0","Type":"ContainerDied","Data":"3cb75690c137c99a5f0cc02d84626e577dc6fd424e07967662f345473656e3da"} Jul 11 23:37:16.802005 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:16.801922 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" event={"ID":"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0","Type":"ContainerStarted","Data":"e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc"} Jul 11 23:37:17.808927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.808896 2580 generic.go:356] "Generic (PLEG): container finished" podID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerID="84b8ec633c3749a1ba4f1bf37101713da8740d577a604fd13491d17ed2239fe6" exitCode=0 Jul 11 23:37:17.809332 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.808983 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" event={"ID":"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda","Type":"ContainerDied","Data":"84b8ec633c3749a1ba4f1bf37101713da8740d577a604fd13491d17ed2239fe6"} Jul 11 23:37:17.810831 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.810809 2580 generic.go:356] "Generic (PLEG): container finished" podID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerID="37912ae46409e07f65746ca6c02db350a93907c381da904684aa5cc067224f24" exitCode=0 Jul 11 23:37:17.810951 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.810897 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" event={"ID":"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0","Type":"ContainerDied","Data":"37912ae46409e07f65746ca6c02db350a93907c381da904684aa5cc067224f24"} Jul 11 23:37:17.812844 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.812821 2580 generic.go:356] "Generic (PLEG): container finished" podID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerID="7217d1435752daaa16c0ba68c1d011ee702398118426e52baf8051e1c7d62101" exitCode=0 Jul 11 23:37:17.812954 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.812842 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" event={"ID":"c3a1f993-3bb8-4655-831d-6621fd9a1078","Type":"ContainerDied","Data":"7217d1435752daaa16c0ba68c1d011ee702398118426e52baf8051e1c7d62101"} Jul 11 23:37:17.815155 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.815137 2580 generic.go:356] "Generic (PLEG): container finished" podID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerID="4c809bcd2eded2083853bcf6080f4e9e359d663f566086ef25b37bd8dffa4759" exitCode=0 Jul 11 23:37:17.815233 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:17.815193 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" event={"ID":"7093ffb8-9b32-45c0-ac31-9bef83bca313","Type":"ContainerDied","Data":"4c809bcd2eded2083853bcf6080f4e9e359d663f566086ef25b37bd8dffa4759"} Jul 11 23:37:18.822316 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.822279 2580 generic.go:356] "Generic (PLEG): container finished" podID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerID="3344c1d0e923a4763adf4d346177b581c96c2818c42bf4f17fc36802b418ba31" exitCode=0 Jul 11 23:37:18.822744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.822369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" event={"ID":"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0","Type":"ContainerDied","Data":"3344c1d0e923a4763adf4d346177b581c96c2818c42bf4f17fc36802b418ba31"} Jul 11 23:37:18.824406 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.824383 2580 generic.go:356] "Generic (PLEG): container finished" podID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerID="dc752a1c07f3a380c603372ded6e0c83c1edc81aa7f25c62c74aba4101a1b8bc" exitCode=0 Jul 11 23:37:18.824514 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.824455 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" event={"ID":"c3a1f993-3bb8-4655-831d-6621fd9a1078","Type":"ContainerDied","Data":"dc752a1c07f3a380c603372ded6e0c83c1edc81aa7f25c62c74aba4101a1b8bc"} Jul 11 23:37:18.826509 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.826488 2580 generic.go:356] "Generic (PLEG): container finished" podID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerID="bc73b4a9b8ea2416597ef149f0e851fc63bdd2156c233aaac1e5d71ecd74d64f" exitCode=0 Jul 11 23:37:18.826597 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.826540 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" event={"ID":"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda","Type":"ContainerDied","Data":"bc73b4a9b8ea2416597ef149f0e851fc63bdd2156c233aaac1e5d71ecd74d64f"} Jul 11 23:37:18.959161 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:18.959135 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:19.044417 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.044383 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util\") pod \"7093ffb8-9b32-45c0-ac31-9bef83bca313\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " Jul 11 23:37:19.044417 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.044416 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle\") pod \"7093ffb8-9b32-45c0-ac31-9bef83bca313\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " Jul 11 23:37:19.044689 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.044458 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qkxnk\" (UniqueName: \"kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk\") pod \"7093ffb8-9b32-45c0-ac31-9bef83bca313\" (UID: \"7093ffb8-9b32-45c0-ac31-9bef83bca313\") " Jul 11 23:37:19.045033 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.045004 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle" (OuterVolumeSpecName: "bundle") pod "7093ffb8-9b32-45c0-ac31-9bef83bca313" (UID: "7093ffb8-9b32-45c0-ac31-9bef83bca313"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:19.046850 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.046827 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk" (OuterVolumeSpecName: "kube-api-access-qkxnk") pod "7093ffb8-9b32-45c0-ac31-9bef83bca313" (UID: "7093ffb8-9b32-45c0-ac31-9bef83bca313"). InnerVolumeSpecName "kube-api-access-qkxnk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:37:19.049869 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.049837 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util" (OuterVolumeSpecName: "util") pod "7093ffb8-9b32-45c0-ac31-9bef83bca313" (UID: "7093ffb8-9b32-45c0-ac31-9bef83bca313"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:19.145445 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.145363 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:19.145445 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.145392 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/7093ffb8-9b32-45c0-ac31-9bef83bca313-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:19.145445 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.145402 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qkxnk\" (UniqueName: \"kubernetes.io/projected/7093ffb8-9b32-45c0-ac31-9bef83bca313-kube-api-access-qkxnk\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:19.833721 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.833671 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" Jul 11 23:37:19.833721 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.833708 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz" event={"ID":"7093ffb8-9b32-45c0-ac31-9bef83bca313","Type":"ContainerDied","Data":"7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9"} Jul 11 23:37:19.834174 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.833759 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="7354cca6a0961474031fec81b308dd40229d63ca313c890be49a4925ab3569e9" Jul 11 23:37:19.986301 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:19.986261 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:20.028398 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.028373 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:20.031417 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.031395 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:20.052949 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.052922 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle\") pod \"c3a1f993-3bb8-4655-831d-6621fd9a1078\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " Jul 11 23:37:20.053112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.052956 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util\") pod \"c3a1f993-3bb8-4655-831d-6621fd9a1078\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " Jul 11 23:37:20.053112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.052972 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util\") pod \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " Jul 11 23:37:20.053112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.052994 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rfclr\" (UniqueName: \"kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr\") pod \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " Jul 11 23:37:20.053112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053058 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle\") pod \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " Jul 11 23:37:20.053112 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053090 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kwf9j\" (UniqueName: \"kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j\") pod \"c3a1f993-3bb8-4655-831d-6621fd9a1078\" (UID: \"c3a1f993-3bb8-4655-831d-6621fd9a1078\") " Jul 11 23:37:20.053368 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053124 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util\") pod \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\" (UID: \"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda\") " Jul 11 23:37:20.053368 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053177 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-74kff\" (UniqueName: \"kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff\") pod \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " Jul 11 23:37:20.053368 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053206 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle\") pod \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\" (UID: \"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0\") " Jul 11 23:37:20.053798 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053557 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle" (OuterVolumeSpecName: "bundle") pod "c3a1f993-3bb8-4655-831d-6621fd9a1078" (UID: "c3a1f993-3bb8-4655-831d-6621fd9a1078"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.053908 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.053807 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.054391 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.054368 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle" (OuterVolumeSpecName: "bundle") pod "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" (UID: "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.055812 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.055786 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr" (OuterVolumeSpecName: "kube-api-access-rfclr") pod "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" (UID: "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda"). InnerVolumeSpecName "kube-api-access-rfclr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:37:20.056051 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.055974 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j" (OuterVolumeSpecName: "kube-api-access-kwf9j") pod "c3a1f993-3bb8-4655-831d-6621fd9a1078" (UID: "c3a1f993-3bb8-4655-831d-6621fd9a1078"). InnerVolumeSpecName "kube-api-access-kwf9j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:37:20.056460 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.056418 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle" (OuterVolumeSpecName: "bundle") pod "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" (UID: "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.056645 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.056610 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff" (OuterVolumeSpecName: "kube-api-access-74kff") pod "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" (UID: "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0"). InnerVolumeSpecName "kube-api-access-74kff". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:37:20.060133 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.059520 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util" (OuterVolumeSpecName: "util") pod "c3a1f993-3bb8-4655-831d-6621fd9a1078" (UID: "c3a1f993-3bb8-4655-831d-6621fd9a1078"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.062171 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.062117 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util" (OuterVolumeSpecName: "util") pod "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" (UID: "10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.063715 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.063684 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util" (OuterVolumeSpecName: "util") pod "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" (UID: "4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.154921 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.154966 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kwf9j\" (UniqueName: \"kubernetes.io/projected/c3a1f993-3bb8-4655-831d-6621fd9a1078-kube-api-access-kwf9j\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.154977 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.154986 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-74kff\" (UniqueName: \"kubernetes.io/projected/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-kube-api-access-74kff\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.154995 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.155003 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/c3a1f993-3bb8-4655-831d-6621fd9a1078-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.155010 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0-util\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.155022 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.155018 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rfclr\" (UniqueName: \"kubernetes.io/projected/4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda-kube-api-access-rfclr\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:37:20.840209 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.840180 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" Jul 11 23:37:20.840598 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.840203 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd" event={"ID":"c3a1f993-3bb8-4655-831d-6621fd9a1078","Type":"ContainerDied","Data":"1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37"} Jul 11 23:37:20.840598 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.840284 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="1864118f4f4ee896f1cdb356fc74cefb062b9df6810ae2b8c5f3668babd1ac37" Jul 11 23:37:20.842310 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.842234 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" Jul 11 23:37:20.842310 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.842246 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr" event={"ID":"4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda","Type":"ContainerDied","Data":"2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa"} Jul 11 23:37:20.842505 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.842319 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="2afc0d2d1a3a1a31fd3346f259999314092ff9b3a2d49482be522e4eebc481aa" Jul 11 23:37:20.844613 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.844588 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" event={"ID":"10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0","Type":"ContainerDied","Data":"e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc"} Jul 11 23:37:20.844706 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.844628 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e59273c8f368de54c0f65600ad4f34e47c4ed0945e1371e9ac6295302cad92dc" Jul 11 23:37:20.844759 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:20.844743 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c" Jul 11 23:37:35.883212 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883172 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4"] Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883531 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883543 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883551 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883557 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883572 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883578 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883584 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883591 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883598 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883603 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883612 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883617 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883629 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883635 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="util" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883641 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883646 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883654 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883659 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="pull" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883665 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883671 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883678 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883683 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883690 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="extract" Jul 11 23:37:35.883744 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883695 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="extract" Jul 11 23:37:35.884483 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883792 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0" containerName="extract" Jul 11 23:37:35.884483 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883803 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c3a1f993-3bb8-4655-831d-6621fd9a1078" containerName="extract" Jul 11 23:37:35.884483 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883810 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7093ffb8-9b32-45c0-ac31-9bef83bca313" containerName="extract" Jul 11 23:37:35.884483 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.883817 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda" containerName="extract" Jul 11 23:37:35.888118 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.888096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:35.890146 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.890119 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-kk9jd\"" Jul 11 23:37:35.890255 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.890199 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 11 23:37:35.890255 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.890208 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 11 23:37:35.891060 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.891048 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 11 23:37:35.891121 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.891089 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 11 23:37:35.894336 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.894284 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4"] Jul 11 23:37:35.996865 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.996825 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nv2pj\" (UniqueName: \"kubernetes.io/projected/c53eb2ab-ed74-4846-8d11-5d547275a91c-kube-api-access-nv2pj\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:35.996865 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.996871 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c53eb2ab-ed74-4846-8d11-5d547275a91c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:35.997072 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:35.996942 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c53eb2ab-ed74-4846-8d11-5d547275a91c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.098323 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.098282 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nv2pj\" (UniqueName: \"kubernetes.io/projected/c53eb2ab-ed74-4846-8d11-5d547275a91c-kube-api-access-nv2pj\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.098521 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.098343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c53eb2ab-ed74-4846-8d11-5d547275a91c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.098521 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.098407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c53eb2ab-ed74-4846-8d11-5d547275a91c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.098983 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.098960 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/c53eb2ab-ed74-4846-8d11-5d547275a91c-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.101035 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.101006 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/c53eb2ab-ed74-4846-8d11-5d547275a91c-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.106495 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.106452 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nv2pj\" (UniqueName: \"kubernetes.io/projected/c53eb2ab-ed74-4846-8d11-5d547275a91c-kube-api-access-nv2pj\") pod \"kuadrant-console-plugin-66b5774985-kj6d4\" (UID: \"c53eb2ab-ed74-4846-8d11-5d547275a91c\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.198930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.198836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" Jul 11 23:37:36.331985 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.331957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4"] Jul 11 23:37:36.333958 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:36.333931 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc53eb2ab_ed74_4846_8d11_5d547275a91c.slice/crio-69f2c2cfeded8e382b670c803d04e56813505ee1b86eec1bc86ffed10664fb61 WatchSource:0}: Error finding container 69f2c2cfeded8e382b670c803d04e56813505ee1b86eec1bc86ffed10664fb61: Status 404 returned error can't find the container with id 69f2c2cfeded8e382b670c803d04e56813505ee1b86eec1bc86ffed10664fb61 Jul 11 23:37:36.928719 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:36.928682 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" event={"ID":"c53eb2ab-ed74-4846-8d11-5d547275a91c","Type":"ContainerStarted","Data":"69f2c2cfeded8e382b670c803d04e56813505ee1b86eec1bc86ffed10664fb61"} Jul 11 23:37:43.335334 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.335301 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-786d79c5b5-s4tzk"] Jul 11 23:37:43.340668 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.340632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.352241 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.351728 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786d79c5b5-s4tzk"] Jul 11 23:37:43.475544 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475455 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-console-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475544 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475545 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-oauth-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475786 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475581 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-trusted-ca-bundle\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475786 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475606 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjrn6\" (UniqueName: \"kubernetes.io/projected/65c156a7-94da-45f3-a438-634545760468-kube-api-access-xjrn6\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475786 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475663 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-service-ca\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475786 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475754 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-oauth-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.475945 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.475809 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576450 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576391 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-console-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576652 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576458 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-oauth-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576652 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576516 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-trusted-ca-bundle\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576652 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576540 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xjrn6\" (UniqueName: \"kubernetes.io/projected/65c156a7-94da-45f3-a438-634545760468-kube-api-access-xjrn6\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576652 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576598 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-service-ca\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576861 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576655 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-oauth-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.576861 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.576688 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.577287 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.577250 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-console-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.577412 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.577371 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-service-ca\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.577531 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.577504 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-trusted-ca-bundle\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.578016 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.577995 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/65c156a7-94da-45f3-a438-634545760468-oauth-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.579241 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.579218 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-oauth-config\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.579490 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.579447 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/65c156a7-94da-45f3-a438-634545760468-console-serving-cert\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.585496 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.585422 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjrn6\" (UniqueName: \"kubernetes.io/projected/65c156a7-94da-45f3-a438-634545760468-kube-api-access-xjrn6\") pod \"console-786d79c5b5-s4tzk\" (UID: \"65c156a7-94da-45f3-a438-634545760468\") " pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:43.655826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:43.655789 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:37:57.702199 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:57.702173 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-786d79c5b5-s4tzk"] Jul 11 23:37:57.704143 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:37:57.704123 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65c156a7_94da_45f3_a438_634545760468.slice/crio-40d8479e81d5a69507d967127132d3acba76b48bfc54aa5e8630f796159bee0b WatchSource:0}: Error finding container 40d8479e81d5a69507d967127132d3acba76b48bfc54aa5e8630f796159bee0b: Status 404 returned error can't find the container with id 40d8479e81d5a69507d967127132d3acba76b48bfc54aa5e8630f796159bee0b Jul 11 23:37:58.062309 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:58.062262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" event={"ID":"c53eb2ab-ed74-4846-8d11-5d547275a91c","Type":"ContainerStarted","Data":"ea3d285dd65b86d5e10d4bcf87db2c1fac160838eb86a72619904db4f40503fc"} Jul 11 23:37:58.063928 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:58.063900 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786d79c5b5-s4tzk" event={"ID":"65c156a7-94da-45f3-a438-634545760468","Type":"ContainerStarted","Data":"2b23dcc5c1d6d434ae86398860f3dfdc46fdba385568b7c5f3a43f6652c62060"} Jul 11 23:37:58.064039 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:58.063944 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-786d79c5b5-s4tzk" event={"ID":"65c156a7-94da-45f3-a438-634545760468","Type":"ContainerStarted","Data":"40d8479e81d5a69507d967127132d3acba76b48bfc54aa5e8630f796159bee0b"} Jul 11 23:37:58.103671 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:58.103604 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-kj6d4" podStartSLOduration=1.794679232 podStartE2EDuration="23.103579753s" totalImagesPullingTime="21.308900521s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:37:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:37:58.101854032 +0000 UTC m=+397.853601568" watchObservedRunningTime="2026-07-11 23:37:58.103579753 +0000 UTC m=+397.855327257" Jul 11 23:37:58.130616 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:37:58.130555 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-786d79c5b5-s4tzk" podStartSLOduration=15.13053357 podStartE2EDuration="15.13053357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:37:43 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:37:58.127673789 +0000 UTC m=+397.879421316" watchObservedRunningTime="2026-07-11 23:37:58.13053357 +0000 UTC m=+397.882281073" Jul 11 23:38:03.656892 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:03.656852 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:38:03.657250 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:03.656916 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:38:03.661727 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:03.661704 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:38:04.100359 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:04.100332 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-786d79c5b5-s4tzk" Jul 11 23:38:04.144373 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:04.144340 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:38:19.070963 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.070923 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:38:19.131441 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.131410 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:38:19.131441 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.131445 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:38:19.131684 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.131548 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.133726 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.133704 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 11 23:38:19.212855 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.212821 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqtsw\" (UniqueName: \"kubernetes.io/projected/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-kube-api-access-nqtsw\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.213045 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.212875 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-config-file\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.314104 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.314065 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqtsw\" (UniqueName: \"kubernetes.io/projected/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-kube-api-access-nqtsw\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.314288 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.314119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-config-file\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.314791 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.314772 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-config-file\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.322048 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.321991 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqtsw\" (UniqueName: \"kubernetes.io/projected/0c131ed1-3aee-4a03-8ee2-3005cfd1d46b-kube-api-access-nqtsw\") pod \"limitador-limitador-54d6d988d5-5gfdw\" (UID: \"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.442253 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.442211 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:19.599460 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.599381 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:38:19.602662 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:38:19.602634 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c131ed1_3aee_4a03_8ee2_3005cfd1d46b.slice/crio-a44ed50a1ffc7696c6511846d04fb943064ad820d70a07703387e457c64960df WatchSource:0}: Error finding container a44ed50a1ffc7696c6511846d04fb943064ad820d70a07703387e457c64960df: Status 404 returned error can't find the container with id a44ed50a1ffc7696c6511846d04fb943064ad820d70a07703387e457c64960df Jul 11 23:38:19.663133 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.663103 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:19.692723 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.692687 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:19.692877 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.692791 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:19.695091 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.695064 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hcq9d\"" Jul 11 23:38:19.819443 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.819412 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hmpqx\" (UniqueName: \"kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx\") pod \"authorino-75d556bd77-vhr5w\" (UID: \"b3f7dd58-45b7-49d9-84a0-b8d245010537\") " pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:19.920504 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.920418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hmpqx\" (UniqueName: \"kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx\") pod \"authorino-75d556bd77-vhr5w\" (UID: \"b3f7dd58-45b7-49d9-84a0-b8d245010537\") " pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:19.929244 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:19.929213 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hmpqx\" (UniqueName: \"kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx\") pod \"authorino-75d556bd77-vhr5w\" (UID: \"b3f7dd58-45b7-49d9-84a0-b8d245010537\") " pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:20.002444 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:20.002415 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:20.187233 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:20.187109 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" event={"ID":"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b","Type":"ContainerStarted","Data":"a44ed50a1ffc7696c6511846d04fb943064ad820d70a07703387e457c64960df"} Jul 11 23:38:20.213857 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:20.213831 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:20.216792 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:38:20.216757 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb3f7dd58_45b7_49d9_84a0_b8d245010537.slice/crio-569ede6e588cfb662023d7e745ef79d92439aebbbce057f7c1191bf94aba8765 WatchSource:0}: Error finding container 569ede6e588cfb662023d7e745ef79d92439aebbbce057f7c1191bf94aba8765: Status 404 returned error can't find the container with id 569ede6e588cfb662023d7e745ef79d92439aebbbce057f7c1191bf94aba8765 Jul 11 23:38:21.195094 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:21.195048 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vhr5w" event={"ID":"b3f7dd58-45b7-49d9-84a0-b8d245010537","Type":"ContainerStarted","Data":"569ede6e588cfb662023d7e745ef79d92439aebbbce057f7c1191bf94aba8765"} Jul 11 23:38:27.236663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:27.236623 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" event={"ID":"0c131ed1-3aee-4a03-8ee2-3005cfd1d46b","Type":"ContainerStarted","Data":"1de11630a07fe93565515e121af842d9cb788f556ff4819a9530dcefe87ed3a6"} Jul 11 23:38:27.237154 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:27.236801 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:27.238023 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:27.237995 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vhr5w" event={"ID":"b3f7dd58-45b7-49d9-84a0-b8d245010537","Type":"ContainerStarted","Data":"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739"} Jul 11 23:38:27.252673 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:27.252625 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" podStartSLOduration=1.488494438 podStartE2EDuration="8.252607839s" totalImagesPullingTime="6.764113401s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:38:27.250670023 +0000 UTC m=+427.002417542" watchObservedRunningTime="2026-07-11 23:38:27.252607839 +0000 UTC m=+427.004355341" Jul 11 23:38:27.265650 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:27.265593 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-vhr5w" podStartSLOduration=2.2846335 podStartE2EDuration="8.265576883s" totalImagesPullingTime="5.980943383s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:38:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:38:27.262865267 +0000 UTC m=+427.014612791" watchObservedRunningTime="2026-07-11 23:38:27.265576883 +0000 UTC m=+427.017324392" Jul 11 23:38:29.168180 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.168112 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-867c84bccf-tgp42" podUID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" containerName="console" containerID="cri-o://1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649" gracePeriod=15 Jul 11 23:38:29.424895 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.424837 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:38:29.513397 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513340 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513397 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513408 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kbttm\" (UniqueName: \"kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513683 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513541 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513683 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513578 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513683 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513651 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513839 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513694 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513839 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513728 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config\") pod \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\" (UID: \"bfe83135-662f-44ac-bcc4-3766fce5ecf3\") " Jul 11 23:38:29.513930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513844 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:38:29.513990 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.513919 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca" (OuterVolumeSpecName: "service-ca") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:38:29.514165 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.514140 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-trusted-ca-bundle\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.514252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.514163 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:38:29.514252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.514169 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.514252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.514173 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config" (OuterVolumeSpecName: "console-config") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 11 23:38:29.515803 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.515779 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:38:29.515803 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.515793 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm" (OuterVolumeSpecName: "kube-api-access-kbttm") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "kube-api-access-kbttm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:38:29.515927 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.515820 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "bfe83135-662f-44ac-bcc4-3766fce5ecf3" (UID: "bfe83135-662f-44ac-bcc4-3766fce5ecf3"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 11 23:38:29.614768 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.614736 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.614768 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.614760 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-oauth-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.614768 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.614770 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-oauth-serving-cert\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.614977 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.614780 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/bfe83135-662f-44ac-bcc4-3766fce5ecf3-console-config\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:29.614977 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:29.614789 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kbttm\" (UniqueName: \"kubernetes.io/projected/bfe83135-662f-44ac-bcc4-3766fce5ecf3-kube-api-access-kbttm\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:30.255177 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.255148 2580 generic.go:356] "Generic (PLEG): container finished" podID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" containerID="1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649" exitCode=0 Jul 11 23:38:30.255586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.255192 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-867c84bccf-tgp42" Jul 11 23:38:30.255586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.255229 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867c84bccf-tgp42" event={"ID":"bfe83135-662f-44ac-bcc4-3766fce5ecf3","Type":"ContainerDied","Data":"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649"} Jul 11 23:38:30.255586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.255281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-867c84bccf-tgp42" event={"ID":"bfe83135-662f-44ac-bcc4-3766fce5ecf3","Type":"ContainerDied","Data":"1aca1b65a953a01f286f0fd69d0ecea097f7cfd7f6ed0193049271d3fc5ce35f"} Jul 11 23:38:30.255586 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.255308 2580 scope.go:117] "RemoveContainer" containerID="1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649" Jul 11 23:38:30.265289 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.265269 2580 scope.go:117] "RemoveContainer" containerID="1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649" Jul 11 23:38:30.265551 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:38:30.265534 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649\": container with ID starting with 1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649 not found: ID does not exist" containerID="1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649" Jul 11 23:38:30.265608 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.265559 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649"} err="failed to get container status \"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649\": rpc error: code = NotFound desc = could not find container \"1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649\": container with ID starting with 1a84111dfa4aa5079f686e430c90808125dfde67108d99b9579f29702e9fb649 not found: ID does not exist" Jul 11 23:38:30.277055 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.277024 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:38:30.279096 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.279076 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-867c84bccf-tgp42"] Jul 11 23:38:30.788566 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:30.788527 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" path="/var/lib/kubelet/pods/bfe83135-662f-44ac-bcc4-3766fce5ecf3/volumes" Jul 11 23:38:38.245385 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:38.245355 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-5gfdw" Jul 11 23:38:54.062236 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.062202 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:54.062662 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.062406 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-vhr5w" podUID="b3f7dd58-45b7-49d9-84a0-b8d245010537" containerName="authorino" containerID="cri-o://acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739" gracePeriod=30 Jul 11 23:38:54.325779 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.325706 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:54.392333 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.392300 2580 generic.go:356] "Generic (PLEG): container finished" podID="b3f7dd58-45b7-49d9-84a0-b8d245010537" containerID="acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739" exitCode=0 Jul 11 23:38:54.392536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.392341 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vhr5w" event={"ID":"b3f7dd58-45b7-49d9-84a0-b8d245010537","Type":"ContainerDied","Data":"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739"} Jul 11 23:38:54.392536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.392344 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-vhr5w" Jul 11 23:38:54.392536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.392374 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-vhr5w" event={"ID":"b3f7dd58-45b7-49d9-84a0-b8d245010537","Type":"ContainerDied","Data":"569ede6e588cfb662023d7e745ef79d92439aebbbce057f7c1191bf94aba8765"} Jul 11 23:38:54.392536 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.392399 2580 scope.go:117] "RemoveContainer" containerID="acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739" Jul 11 23:38:54.402874 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.402610 2580 scope.go:117] "RemoveContainer" containerID="acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739" Jul 11 23:38:54.404004 ip-10-0-140-223 kubenswrapper[2580]: E0711 23:38:54.403971 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739\": container with ID starting with acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739 not found: ID does not exist" containerID="acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739" Jul 11 23:38:54.404125 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.404010 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739"} err="failed to get container status \"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739\": rpc error: code = NotFound desc = could not find container \"acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739\": container with ID starting with acbd83ff53c3fc986dfe7a15cbc2a5139319438eb5ae7ca7f0fc7a061b944739 not found: ID does not exist" Jul 11 23:38:54.420156 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.420125 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hmpqx\" (UniqueName: \"kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx\") pod \"b3f7dd58-45b7-49d9-84a0-b8d245010537\" (UID: \"b3f7dd58-45b7-49d9-84a0-b8d245010537\") " Jul 11 23:38:54.422151 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.422125 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx" (OuterVolumeSpecName: "kube-api-access-hmpqx") pod "b3f7dd58-45b7-49d9-84a0-b8d245010537" (UID: "b3f7dd58-45b7-49d9-84a0-b8d245010537"). InnerVolumeSpecName "kube-api-access-hmpqx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 11 23:38:54.521464 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.521424 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hmpqx\" (UniqueName: \"kubernetes.io/projected/b3f7dd58-45b7-49d9-84a0-b8d245010537-kube-api-access-hmpqx\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 11 23:38:54.714198 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.714159 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:54.721735 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.721708 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-vhr5w"] Jul 11 23:38:54.791083 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:38:54.791048 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b3f7dd58-45b7-49d9-84a0-b8d245010537" path="/var/lib/kubelet/pods/b3f7dd58-45b7-49d9-84a0-b8d245010537/volumes" Jul 11 23:39:34.611563 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:39:34.611528 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:39:53.625745 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:39:53.625707 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:40:04.115631 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:04.115598 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:40:16.931023 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:16.930940 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:40:25.219571 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219532 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh"] Jul 11 23:40:25.219929 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219879 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" containerName="console" Jul 11 23:40:25.219929 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219889 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" containerName="console" Jul 11 23:40:25.219929 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219913 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b3f7dd58-45b7-49d9-84a0-b8d245010537" containerName="authorino" Jul 11 23:40:25.219929 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219919 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="b3f7dd58-45b7-49d9-84a0-b8d245010537" containerName="authorino" Jul 11 23:40:25.220281 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219974 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="bfe83135-662f-44ac-bcc4-3766fce5ecf3" containerName="console" Jul 11 23:40:25.220281 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.219982 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="b3f7dd58-45b7-49d9-84a0-b8d245010537" containerName="authorino" Jul 11 23:40:25.223140 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.223122 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.226127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.226106 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-kq42x\"" Jul 11 23:40:25.226252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.226131 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 11 23:40:25.226252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.226131 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 11 23:40:25.226252 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.226107 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 11 23:40:25.231308 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.231279 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh"] Jul 11 23:40:25.245427 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245393 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7dxwq\" (UniqueName: \"kubernetes.io/projected/4158fcb0-42c5-49da-bc22-0e554c16d5fb-kube-api-access-7dxwq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.245427 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245429 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.245658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245458 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.245658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245503 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.245658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245563 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.245658 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.245602 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346242 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346249 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7dxwq\" (UniqueName: \"kubernetes.io/projected/4158fcb0-42c5-49da-bc22-0e554c16d5fb-kube-api-access-7dxwq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346328 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346681 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346483 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346681 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346531 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346681 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346661 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346822 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346742 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.346822 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.346800 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.348446 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.348428 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/4158fcb0-42c5-49da-bc22-0e554c16d5fb-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.348832 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.348816 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/4158fcb0-42c5-49da-bc22-0e554c16d5fb-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.354241 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.354215 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7dxwq\" (UniqueName: \"kubernetes.io/projected/4158fcb0-42c5-49da-bc22-0e554c16d5fb-kube-api-access-7dxwq\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh\" (UID: \"4158fcb0-42c5-49da-bc22-0e554c16d5fb\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.535492 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.535431 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:25.668826 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.668801 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh"] Jul 11 23:40:25.670462 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:40:25.670431 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4158fcb0_42c5_49da_bc22_0e554c16d5fb.slice/crio-6b7608adf001b7cb647bd7bcdf6e4e77d9e07f26b4521648074f2638acf4f00b WatchSource:0}: Error finding container 6b7608adf001b7cb647bd7bcdf6e4e77d9e07f26b4521648074f2638acf4f00b: Status 404 returned error can't find the container with id 6b7608adf001b7cb647bd7bcdf6e4e77d9e07f26b4521648074f2638acf4f00b Jul 11 23:40:25.880747 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:25.880656 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" event={"ID":"4158fcb0-42c5-49da-bc22-0e554c16d5fb","Type":"ContainerStarted","Data":"6b7608adf001b7cb647bd7bcdf6e4e77d9e07f26b4521648074f2638acf4f00b"} Jul 11 23:40:26.819676 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:26.819635 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:40:26.887547 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:26.887506 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" event={"ID":"4158fcb0-42c5-49da-bc22-0e554c16d5fb","Type":"ContainerStarted","Data":"3d00fff40658cd3723e0b290d99d5ce22ce7f988d128c225bdab513f1da6bb50"} Jul 11 23:40:26.904434 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:26.904381 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" podStartSLOduration=0.911450828 podStartE2EDuration="1.904365035s" totalImagesPullingTime="992.914207ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:40:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:40:26.9018725 +0000 UTC m=+546.653620054" watchObservedRunningTime="2026-07-11 23:40:26.904365035 +0000 UTC m=+546.656112538" Jul 11 23:40:35.536253 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:35.536212 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:35.536646 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:35.536292 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:35.551625 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:35.551597 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:35.929136 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:35.929050 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:40:35.945159 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:35.945130 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh" Jul 11 23:40:43.225227 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:40:43.225189 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:41:20.725222 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:20.725195 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:41:20.725705 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:20.725195 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:41:47.767256 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.767212 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7b954d4f86-w5nbl"] Jul 11 23:41:47.770687 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.770669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:47.773678 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.773652 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jul 11 23:41:47.773796 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.773656 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-hcq9d\"" Jul 11 23:41:47.778072 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.777884 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b954d4f86-w5nbl"] Jul 11 23:41:47.893552 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.893511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ba8e080-f88f-4de4-a251-fe177e308638-tls-cert\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:47.893757 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.893609 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qwgw7\" (UniqueName: \"kubernetes.io/projected/2ba8e080-f88f-4de4-a251-fe177e308638-kube-api-access-qwgw7\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:47.994510 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.994443 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qwgw7\" (UniqueName: \"kubernetes.io/projected/2ba8e080-f88f-4de4-a251-fe177e308638-kube-api-access-qwgw7\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:47.994727 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.994588 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ba8e080-f88f-4de4-a251-fe177e308638-tls-cert\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:47.997031 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:47.997006 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/2ba8e080-f88f-4de4-a251-fe177e308638-tls-cert\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:48.003048 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:48.003021 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qwgw7\" (UniqueName: \"kubernetes.io/projected/2ba8e080-f88f-4de4-a251-fe177e308638-kube-api-access-qwgw7\") pod \"authorino-7b954d4f86-w5nbl\" (UID: \"2ba8e080-f88f-4de4-a251-fe177e308638\") " pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:48.081023 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:48.080938 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7b954d4f86-w5nbl" Jul 11 23:41:48.219718 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:48.219693 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7b954d4f86-w5nbl"] Jul 11 23:41:48.221840 ip-10-0-140-223 kubenswrapper[2580]: W0711 23:41:48.221812 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2ba8e080_f88f_4de4_a251_fe177e308638.slice/crio-23ee4c0459c9ae0b3094228035246534bba8ba01ab3c88dc62e7abe0bde224e7 WatchSource:0}: Error finding container 23ee4c0459c9ae0b3094228035246534bba8ba01ab3c88dc62e7abe0bde224e7: Status 404 returned error can't find the container with id 23ee4c0459c9ae0b3094228035246534bba8ba01ab3c88dc62e7abe0bde224e7 Jul 11 23:41:48.322155 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:48.322124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b954d4f86-w5nbl" event={"ID":"2ba8e080-f88f-4de4-a251-fe177e308638","Type":"ContainerStarted","Data":"23ee4c0459c9ae0b3094228035246534bba8ba01ab3c88dc62e7abe0bde224e7"} Jul 11 23:41:49.330820 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:49.330776 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7b954d4f86-w5nbl" event={"ID":"2ba8e080-f88f-4de4-a251-fe177e308638","Type":"ContainerStarted","Data":"4407d5dd9100ea937d0595a8c86ba32602742d60be292f8f7cd645bc2bb5452d"} Jul 11 23:41:49.348958 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:41:49.348907 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-7b954d4f86-w5nbl" podStartSLOduration=2.001932042 podStartE2EDuration="2.348889064s" totalImagesPullingTime="346.957022ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-11 23:41:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-11 23:41:49.348153699 +0000 UTC m=+629.099901203" watchObservedRunningTime="2026-07-11 23:41:49.348889064 +0000 UTC m=+629.100636566" Jul 11 23:42:17.313743 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:42:17.313704 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:42:26.619224 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:42:26.619192 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:42:37.517840 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:42:37.517800 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:42:46.324254 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:42:46.324210 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:42:57.121768 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:42:57.121730 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:43:25.311716 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:43:25.311641 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:43:36.315590 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:43:36.315556 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:09.817205 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:09.817171 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:19.519701 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:19.519659 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:28.214930 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:28.214897 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:37.323695 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:37.323654 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:47.516299 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:47.516262 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:44:56.516784 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:44:56.516741 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:45:08.719086 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:45:08.719047 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:45:18.314199 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:45:18.314161 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:45:40.517893 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:45:40.517852 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:45:51.017698 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:45:51.017662 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:46:13.230535 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:13.230411 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:46:20.759553 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:20.759525 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:46:20.762210 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:20.762187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:46:23.517017 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:23.516980 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:46:38.517048 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:38.517004 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:46:50.423858 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:46:50.423822 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:47:00.028491 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:47:00.028329 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:47:09.220127 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:47:09.220091 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:47:18.422463 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:47:18.422428 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:04.118120 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:04.118076 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:12.722204 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:12.722125 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:25.714185 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:25.714148 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:36.719033 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:36.718996 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:47.512328 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:47.512284 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:49:58.818754 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:49:58.818720 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:50:07.717663 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:50:07.717623 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:50:18.518579 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:50:18.518542 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:50:29.019406 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:50:29.019362 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:50:37.920708 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:50:37.920671 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:50:48.318762 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:50:48.318715 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:51:10.521454 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:10.521416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:51:19.722454 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:19.722416 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:51:20.796992 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:20.796965 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:51:20.798305 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:20.798284 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:51:31.916010 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:31.915973 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:51:39.617390 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:39.617357 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:51:51.010501 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:51:51.010448 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:52:05.319695 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:52:05.319661 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:52:20.023459 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:52:20.023417 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:52:29.312675 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:52:29.312639 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:52:42.012916 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:52:42.012881 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:52:51.016684 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:52:51.016649 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:03.911357 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:03.911322 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:12.925023 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:12.924991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:24.217938 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:24.217904 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:33.717960 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:33.717925 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:44.620729 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:44.620693 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:53:53.621822 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:53:53.621789 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:54:06.418211 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:54:06.418172 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:54:18.915842 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:54:18.915803 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:54:31.519883 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:54:31.519840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:54:40.416552 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:54:40.416514 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:54:52.525837 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:54:52.525801 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:55:01.531361 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:55:01.531321 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:55:10.617998 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:55:10.617961 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:56:15.913733 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:56:15.913693 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:56:20.831917 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:56:20.831891 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:56:20.834992 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:56:20.834970 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 11 23:56:34.614811 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:56:34.614770 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:57:37.119800 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:57:37.119759 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:57:49.514437 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:57:49.514400 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:59:05.021548 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:59:05.021497 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 11 23:59:21.617146 ip-10-0-140-223 kubenswrapper[2580]: I0711 23:59:21.617108 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:00:00.138407 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.138362 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-pruner-29730240-v24zp"] Jul 12 00:00:00.142051 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.142024 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.144242 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.144211 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"serviceca\"" Jul 12 00:00:00.144405 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.144211 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"pruner-dockercfg-z7wbx\"" Jul 12 00:00:00.152157 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.152131 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29730240-v24zp"] Jul 12 00:00:00.275121 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.268705 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwkvl\" (UniqueName: \"kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.275121 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.268788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.369997 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.369951 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwkvl\" (UniqueName: \"kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.370180 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.370006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.370628 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.370609 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.377371 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.377343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwkvl\" (UniqueName: \"kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl\") pod \"image-pruner-29730240-v24zp\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.470598 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.470502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:00.615450 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:00.615422 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-pruner-29730240-v24zp"] Jul 12 00:00:00.617597 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:00:00.617569 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08260403_1024_4a91_8c28_ac844874192b.slice/crio-6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf WatchSource:0}: Error finding container 6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf: Status 404 returned error can't find the container with id 6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf Jul 12 00:00:01.234770 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:01.234735 2580 generic.go:356] "Generic (PLEG): container finished" podID="08260403-1024-4a91-8c28-ac844874192b" containerID="894841be54850ce24e0c111244acf8deafff1b37adaef108f35d82050fefaf86" exitCode=0 Jul 12 00:00:01.235162 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:01.234787 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29730240-v24zp" event={"ID":"08260403-1024-4a91-8c28-ac844874192b","Type":"ContainerDied","Data":"894841be54850ce24e0c111244acf8deafff1b37adaef108f35d82050fefaf86"} Jul 12 00:00:01.235162 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:01.234834 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29730240-v24zp" event={"ID":"08260403-1024-4a91-8c28-ac844874192b","Type":"ContainerStarted","Data":"6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf"} Jul 12 00:00:02.372746 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.372719 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:02.489890 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.489851 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwkvl\" (UniqueName: \"kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl\") pod \"08260403-1024-4a91-8c28-ac844874192b\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " Jul 12 00:00:02.490089 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.489978 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca\") pod \"08260403-1024-4a91-8c28-ac844874192b\" (UID: \"08260403-1024-4a91-8c28-ac844874192b\") " Jul 12 00:00:02.490531 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.490506 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca" (OuterVolumeSpecName: "serviceca") pod "08260403-1024-4a91-8c28-ac844874192b" (UID: "08260403-1024-4a91-8c28-ac844874192b"). InnerVolumeSpecName "serviceca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 00:00:02.492238 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.492183 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl" (OuterVolumeSpecName: "kube-api-access-vwkvl") pod "08260403-1024-4a91-8c28-ac844874192b" (UID: "08260403-1024-4a91-8c28-ac844874192b"). InnerVolumeSpecName "kube-api-access-vwkvl". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:00:02.591637 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.591583 2580 reconciler_common.go:299] "Volume detached for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/08260403-1024-4a91-8c28-ac844874192b-serviceca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:00:02.591637 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:02.591631 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwkvl\" (UniqueName: \"kubernetes.io/projected/08260403-1024-4a91-8c28-ac844874192b-kube-api-access-vwkvl\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:00:03.246812 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:03.246775 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-pruner-29730240-v24zp" event={"ID":"08260403-1024-4a91-8c28-ac844874192b","Type":"ContainerDied","Data":"6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf"} Jul 12 00:00:03.247017 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:03.246832 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="6641b85d7a0251aa347772d990ee6b320ad9d419e915b7f7c79b129756820fcf" Jul 12 00:00:03.247017 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:03.246793 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-pruner-29730240-v24zp" Jul 12 00:00:36.716042 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:00:36.716003 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:01:20.868387 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:01:20.868355 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:01:20.871951 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:01:20.871929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:02:09.713167 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:02:09.713129 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:02:18.419239 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:02:18.419197 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:02:28.621051 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:02:28.621017 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:02:50.413751 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:02:50.413715 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:03:31.116607 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:03:31.116568 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:03:56.114419 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:03:56.114382 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:04:14.320676 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:04:14.320594 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:04:31.523957 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:04:31.523921 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:05:49.317764 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:05:49.317724 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:06:00.718910 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:00.718872 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:06:20.906234 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:20.906200 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:06:20.910662 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:20.910644 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:06:49.129026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.128991 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:06:49.860104 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.860067 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:06:49.860527 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.860511 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08260403-1024-4a91-8c28-ac844874192b" containerName="image-pruner" Jul 12 00:06:49.860527 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.860528 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="08260403-1024-4a91-8c28-ac844874192b" containerName="image-pruner" Jul 12 00:06:49.860624 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.860588 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="08260403-1024-4a91-8c28-ac844874192b" containerName="image-pruner" Jul 12 00:06:49.863908 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.863887 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.866425 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.866408 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-e2e-inf-b-ad0409-serving-cert\"" Jul 12 00:06:49.866586 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.866563 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 00:06:49.867192 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.867171 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-dockercfg-6n9rc\"" Jul 12 00:06:49.867192 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.867189 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 12 00:06:49.874703 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.874682 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:06:49.893240 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.893214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.893343 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.893270 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nc7th\" (UniqueName: \"kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.893343 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.893323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.994157 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.994128 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.994296 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.994201 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nc7th\" (UniqueName: \"kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.994296 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.994239 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:49.994398 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:06:49.994382 2580 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-serving-cert: secret "maas-api-e2e-inf-b-ad0409-serving-cert" not found Jul 12 00:06:49.994500 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:06:49.994481 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls podName:17b5868b-9a23-4581-a68d-757b997c59e4 nodeName:}" failed. No retries permitted until 2026-07-12 00:06:50.494444879 +0000 UTC m=+2130.246192383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls") pod "maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" (UID: "17b5868b-9a23-4581-a68d-757b997c59e4") : secret "maas-api-e2e-inf-b-ad0409-serving-cert" not found Jul 12 00:06:49.994890 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:49.994871 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:50.002895 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.002870 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nc7th\" (UniqueName: \"kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:50.499988 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.499958 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:50.502296 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.502267 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") pod \"maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:50.776326 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.776235 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:50.915980 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.915948 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:06:50.917561 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:06:50.917534 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod17b5868b_9a23_4581_a68d_757b997c59e4.slice/crio-4613e319949a8ae76b09e24e43b6a3b185c52c3134ff88024360b580b9d6e667 WatchSource:0}: Error finding container 4613e319949a8ae76b09e24e43b6a3b185c52c3134ff88024360b580b9d6e667: Status 404 returned error can't find the container with id 4613e319949a8ae76b09e24e43b6a3b185c52c3134ff88024360b580b9d6e667 Jul 12 00:06:50.919270 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:50.919252 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 00:06:51.439041 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:51.439001 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" event={"ID":"17b5868b-9a23-4581-a68d-757b997c59e4","Type":"ContainerStarted","Data":"4613e319949a8ae76b09e24e43b6a3b185c52c3134ff88024360b580b9d6e667"} Jul 12 00:06:53.451223 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:53.451183 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" event={"ID":"17b5868b-9a23-4581-a68d-757b997c59e4","Type":"ContainerStarted","Data":"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210"} Jul 12 00:06:53.451599 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:53.451322 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:53.467966 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:53.467914 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" podStartSLOduration=2.791508026 podStartE2EDuration="4.467899559s" totalImagesPullingTime="1.676391533s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:06:49 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:06:53.465152284 +0000 UTC m=+2133.216899788" watchObservedRunningTime="2026-07-12 00:06:53.467899559 +0000 UTC m=+2133.219647063" Jul 12 00:06:59.461756 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:59.461727 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:06:59.813795 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:06:59.813762 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:07:11.015395 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:07:11.015348 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:07:51.118436 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:07:51.118384 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:08:14.721881 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:14.721847 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:08:26.715993 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:26.715957 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:08:45.975333 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:45.975291 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:08:45.975944 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:45.975516 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" podUID="17b5868b-9a23-4581-a68d-757b997c59e4" containerName="maas-api" containerID="cri-o://38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210" gracePeriod=30 Jul 12 00:08:46.220754 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.220731 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:08:46.331127 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.331035 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nc7th\" (UniqueName: \"kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th\") pod \"17b5868b-9a23-4581-a68d-757b997c59e4\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " Jul 12 00:08:46.331299 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.331131 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca\") pod \"17b5868b-9a23-4581-a68d-757b997c59e4\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " Jul 12 00:08:46.331299 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.331222 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") pod \"17b5868b-9a23-4581-a68d-757b997c59e4\" (UID: \"17b5868b-9a23-4581-a68d-757b997c59e4\") " Jul 12 00:08:46.331535 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.331508 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca" (OuterVolumeSpecName: "openshift-service-ca") pod "17b5868b-9a23-4581-a68d-757b997c59e4" (UID: "17b5868b-9a23-4581-a68d-757b997c59e4"). InnerVolumeSpecName "openshift-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 00:08:46.331616 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.331576 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/17b5868b-9a23-4581-a68d-757b997c59e4-openshift-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:08:46.333333 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.333308 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls" (OuterVolumeSpecName: "maas-api-tls") pod "17b5868b-9a23-4581-a68d-757b997c59e4" (UID: "17b5868b-9a23-4581-a68d-757b997c59e4"). InnerVolumeSpecName "maas-api-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 00:08:46.333422 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.333368 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th" (OuterVolumeSpecName: "kube-api-access-nc7th") pod "17b5868b-9a23-4581-a68d-757b997c59e4" (UID: "17b5868b-9a23-4581-a68d-757b997c59e4"). InnerVolumeSpecName "kube-api-access-nc7th". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:08:46.432947 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.432908 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nc7th\" (UniqueName: \"kubernetes.io/projected/17b5868b-9a23-4581-a68d-757b997c59e4-kube-api-access-nc7th\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:08:46.432947 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:46.432944 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/17b5868b-9a23-4581-a68d-757b997c59e4-maas-api-tls\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:08:47.052231 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.052130 2580 generic.go:356] "Generic (PLEG): container finished" podID="17b5868b-9a23-4581-a68d-757b997c59e4" containerID="38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210" exitCode=0 Jul 12 00:08:47.052231 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.052174 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" Jul 12 00:08:47.052734 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.052174 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" event={"ID":"17b5868b-9a23-4581-a68d-757b997c59e4","Type":"ContainerDied","Data":"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210"} Jul 12 00:08:47.052734 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.052301 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq" event={"ID":"17b5868b-9a23-4581-a68d-757b997c59e4","Type":"ContainerDied","Data":"4613e319949a8ae76b09e24e43b6a3b185c52c3134ff88024360b580b9d6e667"} Jul 12 00:08:47.052734 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.052335 2580 scope.go:117] "RemoveContainer" containerID="38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210" Jul 12 00:08:47.061640 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.061588 2580 scope.go:117] "RemoveContainer" containerID="38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210" Jul 12 00:08:47.061865 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:08:47.061845 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210\": container with ID starting with 38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210 not found: ID does not exist" containerID="38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210" Jul 12 00:08:47.061913 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.061875 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210"} err="failed to get container status \"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210\": rpc error: code = NotFound desc = could not find container \"38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210\": container with ID starting with 38503cff531f5e11c0ee67c286d3541b509150ad2f75b411f8589c5aa01eb210 not found: ID does not exist" Jul 12 00:08:47.068023 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.068000 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:08:47.071317 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:47.071292 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["odh-ai-gateway-infra/maas-api-e2e-inf-b-ad0409-794f88b7fb-khqmq"] Jul 12 00:08:48.788909 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:48.788878 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="17b5868b-9a23-4581-a68d-757b997c59e4" path="/var/lib/kubelet/pods/17b5868b-9a23-4581-a68d-757b997c59e4/volumes" Jul 12 00:08:48.917684 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:08:48.917650 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:09:10.018541 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:10.018502 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:09:21.017605 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:21.017563 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:09:26.317582 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.317534 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn"] Jul 12 00:09:26.318106 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.318087 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="17b5868b-9a23-4581-a68d-757b997c59e4" containerName="maas-api" Jul 12 00:09:26.318192 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.318110 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="17b5868b-9a23-4581-a68d-757b997c59e4" containerName="maas-api" Jul 12 00:09:26.318259 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.318244 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="17b5868b-9a23-4581-a68d-757b997c59e4" containerName="maas-api" Jul 12 00:09:26.321259 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.321238 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:26.324644 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.324622 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 12 00:09:26.324644 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.324637 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-cleanup-dockercfg-9q4fh\"" Jul 12 00:09:26.326868 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.326838 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn"] Jul 12 00:09:26.370858 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.370833 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-665549-cvnjn\" (UID: \"ebabcd0e-09a2-4208-9a67-f16b010a2e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:26.472154 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.472119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-665549-cvnjn\" (UID: \"ebabcd0e-09a2-4208-9a67-f16b010a2e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:26.472755 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.472736 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca\") pod \"maas-api-revoke-keys-e2e-api-a-665549-cvnjn\" (UID: \"ebabcd0e-09a2-4208-9a67-f16b010a2e99\") " pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:26.632787 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.632711 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:26.760047 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:26.760013 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn"] Jul 12 00:09:26.761074 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:09:26.761051 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebabcd0e_09a2_4208_9a67_f16b010a2e99.slice/crio-3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94 WatchSource:0}: Error finding container 3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94: Status 404 returned error can't find the container with id 3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94 Jul 12 00:09:27.274132 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:27.274098 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" event={"ID":"ebabcd0e-09a2-4208-9a67-f16b010a2e99","Type":"ContainerStarted","Data":"3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94"} Jul 12 00:09:30.292447 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:30.292401 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" event={"ID":"ebabcd0e-09a2-4208-9a67-f16b010a2e99","Type":"ContainerStarted","Data":"d778b3afa112e6f341436be050c4fe00c1cb0063a98a90f7b497ada7ae47459b"} Jul 12 00:09:30.307541 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:30.307497 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" podStartSLOduration=1.205045071 podStartE2EDuration="4.307457162s" totalImagesPullingTime="3.102412091s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:09:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:09:30.306305584 +0000 UTC m=+2290.058053087" watchObservedRunningTime="2026-07-12 00:09:30.307457162 +0000 UTC m=+2290.059204665" Jul 12 00:09:31.299341 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:31.299310 2580 generic.go:356] "Generic (PLEG): container finished" podID="ebabcd0e-09a2-4208-9a67-f16b010a2e99" containerID="d778b3afa112e6f341436be050c4fe00c1cb0063a98a90f7b497ada7ae47459b" exitCode=0 Jul 12 00:09:31.299739 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:31.299397 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" event={"ID":"ebabcd0e-09a2-4208-9a67-f16b010a2e99","Type":"ContainerDied","Data":"d778b3afa112e6f341436be050c4fe00c1cb0063a98a90f7b497ada7ae47459b"} Jul 12 00:09:32.441986 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:32.441962 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:32.531554 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:32.531524 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca\") pod \"ebabcd0e-09a2-4208-9a67-f16b010a2e99\" (UID: \"ebabcd0e-09a2-4208-9a67-f16b010a2e99\") " Jul 12 00:09:32.531926 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:32.531900 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca" (OuterVolumeSpecName: "maas-api-service-ca") pod "ebabcd0e-09a2-4208-9a67-f16b010a2e99" (UID: "ebabcd0e-09a2-4208-9a67-f16b010a2e99"). InnerVolumeSpecName "maas-api-service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 12 00:09:32.632908 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:32.632822 2580 reconciler_common.go:299] "Volume detached for volume \"maas-api-service-ca\" (UniqueName: \"kubernetes.io/configmap/ebabcd0e-09a2-4208-9a67-f16b010a2e99-maas-api-service-ca\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:09:33.310822 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:33.310797 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" Jul 12 00:09:33.310988 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:33.310839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-revoke-keys-e2e-api-a-665549-cvnjn" event={"ID":"ebabcd0e-09a2-4208-9a67-f16b010a2e99","Type":"ContainerDied","Data":"3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94"} Jul 12 00:09:33.310988 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:33.310887 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3ce55c3e1dd056dc9b8ab6fc9f72afd33665ee9f84ed49d7bb38797cc6bf3a94" Jul 12 00:09:40.923109 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:09:40.923075 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:10:06.214038 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:10:06.213998 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:10:27.520245 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:10:27.520168 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:10:35.818614 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:10:35.818580 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:10:53.310295 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:10:53.310260 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:11:20.942870 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:11:20.942838 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:11:20.949239 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:11:20.949217 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:11:21.318649 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:11:21.318611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:11:36.120746 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:11:36.120713 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:11:54.614272 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:11:54.614235 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:12:10.020949 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:12:10.020912 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:12:26.924151 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:12:26.924115 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:12:43.821515 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:12:43.821479 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:12:59.516716 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:12:59.516677 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:13:17.316225 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:17.316146 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:13:19.118660 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.118631 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:13:19.119019 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.118982 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebabcd0e-09a2-4208-9a67-f16b010a2e99" containerName="revoke-keys" Jul 12 00:13:19.119019 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.118993 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebabcd0e-09a2-4208-9a67-f16b010a2e99" containerName="revoke-keys" Jul 12 00:13:19.119096 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.119067 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebabcd0e-09a2-4208-9a67-f16b010a2e99" containerName="revoke-keys" Jul 12 00:13:19.122104 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.122084 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.125334 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.125312 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7d1ef\"/\"openshift-service-ca.crt\"" Jul 12 00:13:19.125455 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.125382 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7d1ef\"/\"sub-test-model-a7d1ef-kserve-self-signed-certs\"" Jul 12 00:13:19.125455 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.125393 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-a7d1ef\"/\"default-dockercfg-7tpxz\"" Jul 12 00:13:19.125455 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.125407 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-a7d1ef\"/\"kube-root-ca.crt\"" Jul 12 00:13:19.131645 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.131622 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:13:19.165334 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165302 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.165334 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165333 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.165550 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.165550 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165364 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.165550 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165385 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zb2mn\" (UniqueName: \"kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.165550 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.165426 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266496 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266456 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266496 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266499 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266711 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266515 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266711 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266711 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266665 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zb2mn\" (UniqueName: \"kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266876 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266753 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266876 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266850 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.266997 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.266911 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.267087 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.267068 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.268791 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.268765 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.269008 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.268990 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.273875 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.273856 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zb2mn\" (UniqueName: \"kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn\") pod \"sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.432374 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.432292 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:19.563306 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:19.563282 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:13:19.565126 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:13:19.565098 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod477d9e8a_08e3_4e32_afb5_a98704daf542.slice/crio-a1c5deea018a27b0775763aa2f41598926638b210856257fde318b8b0824d8aa WatchSource:0}: Error finding container a1c5deea018a27b0775763aa2f41598926638b210856257fde318b8b0824d8aa: Status 404 returned error can't find the container with id a1c5deea018a27b0775763aa2f41598926638b210856257fde318b8b0824d8aa Jul 12 00:13:20.545803 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:20.545763 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" event={"ID":"477d9e8a-08e3-4e32-afb5-a98704daf542","Type":"ContainerStarted","Data":"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454"} Jul 12 00:13:20.545803 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:20.545806 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" event={"ID":"477d9e8a-08e3-4e32-afb5-a98704daf542","Type":"ContainerStarted","Data":"a1c5deea018a27b0775763aa2f41598926638b210856257fde318b8b0824d8aa"} Jul 12 00:13:20.564083 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:20.564037 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" podStartSLOduration=1.564022054 podStartE2EDuration="1.564022054s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:13:19 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:13:20.562646057 +0000 UTC m=+2520.314393560" watchObservedRunningTime="2026-07-12 00:13:20.564022054 +0000 UTC m=+2520.315769856" Jul 12 00:13:29.432622 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:29.432583 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:29.432622 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:29.432630 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:29.444991 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:29.444969 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:29.611441 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:29.611396 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:13:32.721630 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:13:32.721596 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:14:27.023374 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:14:27.023333 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:14:43.218008 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:14:43.217964 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:15:01.265212 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:01.265173 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:15:03.913659 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:03.913621 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:15:16.284433 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.284372 2580 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" podUID="477d9e8a-08e3-4e32-afb5-a98704daf542" containerName="main" containerID="cri-o://bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454" gracePeriod=45 Jul 12 00:15:16.535711 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.535653 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:15:16.633840 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633808 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633852 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633873 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633894 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633915 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.633977 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zb2mn\" (UniqueName: \"kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn\") pod \"477d9e8a-08e3-4e32-afb5-a98704daf542\" (UID: \"477d9e8a-08e3-4e32-afb5-a98704daf542\") " Jul 12 00:15:16.634279 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634076 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home" (OuterVolumeSpecName: "home") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 00:15:16.634279 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634102 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache" (OuterVolumeSpecName: "model-cache") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 00:15:16.634279 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634192 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 00:15:16.634436 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634328 2580 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-model-cache\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:16.634436 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634362 2580 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-home\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:16.634436 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.634375 2580 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-tmp-dir\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:16.636111 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.636082 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 12 00:15:16.636215 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.636120 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm" (OuterVolumeSpecName: "dshm") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 12 00:15:16.636215 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.636137 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn" (OuterVolumeSpecName: "kube-api-access-zb2mn") pod "477d9e8a-08e3-4e32-afb5-a98704daf542" (UID: "477d9e8a-08e3-4e32-afb5-a98704daf542"). InnerVolumeSpecName "kube-api-access-zb2mn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:15:16.735701 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.735666 2580 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/477d9e8a-08e3-4e32-afb5-a98704daf542-dshm\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:16.735701 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.735694 2580 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/477d9e8a-08e3-4e32-afb5-a98704daf542-tls-certs\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:16.735701 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:16.735704 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zb2mn\" (UniqueName: \"kubernetes.io/projected/477d9e8a-08e3-4e32-afb5-a98704daf542-kube-api-access-zb2mn\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:15:17.174660 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.174570 2580 generic.go:356] "Generic (PLEG): container finished" podID="477d9e8a-08e3-4e32-afb5-a98704daf542" containerID="bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454" exitCode=0 Jul 12 00:15:17.174660 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.174612 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" Jul 12 00:15:17.174836 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.174655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" event={"ID":"477d9e8a-08e3-4e32-afb5-a98704daf542","Type":"ContainerDied","Data":"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454"} Jul 12 00:15:17.174836 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.174698 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk" event={"ID":"477d9e8a-08e3-4e32-afb5-a98704daf542","Type":"ContainerDied","Data":"a1c5deea018a27b0775763aa2f41598926638b210856257fde318b8b0824d8aa"} Jul 12 00:15:17.174836 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.174714 2580 scope.go:117] "RemoveContainer" containerID="bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454" Jul 12 00:15:17.183494 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.183462 2580 scope.go:117] "RemoveContainer" containerID="bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454" Jul 12 00:15:17.183756 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:15:17.183737 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454\": container with ID starting with bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454 not found: ID does not exist" containerID="bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454" Jul 12 00:15:17.183816 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.183764 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454"} err="failed to get container status \"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454\": rpc error: code = NotFound desc = could not find container \"bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454\": container with ID starting with bc98cdc27c91e6acfb5612f1cc2f61947850b05d7e92f88548c72c2408039454 not found: ID does not exist" Jul 12 00:15:17.191680 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.191660 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:15:17.195040 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:17.195022 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-a7d1ef/sub-test-model-a7d1ef-kserve-57dd8b8655-d9cjk"] Jul 12 00:15:18.788547 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:18.788519 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="477d9e8a-08e3-4e32-afb5-a98704daf542" path="/var/lib/kubelet/pods/477d9e8a-08e3-4e32-afb5-a98704daf542/volumes" Jul 12 00:15:19.022014 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:19.021985 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:15:35.124239 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:35.124199 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:15:50.215542 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:15:50.215504 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:16:05.016060 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:05.016027 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:16:14.924895 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.924814 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783380401072"] Jul 12 00:16:14.925318 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.925180 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="477d9e8a-08e3-4e32-afb5-a98704daf542" containerName="main" Jul 12 00:16:14.925318 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.925191 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="477d9e8a-08e3-4e32-afb5-a98704daf542" containerName="main" Jul 12 00:16:14.925318 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.925254 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="477d9e8a-08e3-4e32-afb5-a98704daf542" containerName="main" Jul 12 00:16:14.928237 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.928216 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:14.930291 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.930272 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-dockercfg-bsmjx\"" Jul 12 00:16:14.933507 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:14.933484 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783380401072"] Jul 12 00:16:15.030850 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.030823 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g22jv\" (UniqueName: \"kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv\") pod \"test-curl-7344-139783380401072\" (UID: \"77477a44-d040-48d0-95da-bde461648995\") " pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:15.131911 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.131883 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g22jv\" (UniqueName: \"kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv\") pod \"test-curl-7344-139783380401072\" (UID: \"77477a44-d040-48d0-95da-bde461648995\") " pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:15.139367 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.139343 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g22jv\" (UniqueName: \"kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv\") pod \"test-curl-7344-139783380401072\" (UID: \"77477a44-d040-48d0-95da-bde461648995\") " pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:15.238166 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.238138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:15.364697 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.364675 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783380401072"] Jul 12 00:16:15.366446 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:15.366417 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod77477a44_d040_48d0_95da_bde461648995.slice/crio-d26337c72266375c58d7f6b2230a7e012d47bb9c010ab8b9f912c3ec639f98a8 WatchSource:0}: Error finding container d26337c72266375c58d7f6b2230a7e012d47bb9c010ab8b9f912c3ec639f98a8: Status 404 returned error can't find the container with id d26337c72266375c58d7f6b2230a7e012d47bb9c010ab8b9f912c3ec639f98a8 Jul 12 00:16:15.367744 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.367723 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 12 00:16:15.488363 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:15.488287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783380401072" event={"ID":"77477a44-d040-48d0-95da-bde461648995","Type":"ContainerStarted","Data":"d26337c72266375c58d7f6b2230a7e012d47bb9c010ab8b9f912c3ec639f98a8"} Jul 12 00:16:16.500624 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:16.500594 2580 generic.go:356] "Generic (PLEG): container finished" podID="77477a44-d040-48d0-95da-bde461648995" containerID="f865b407e4f61012c4e6598374f68b12ce9c8970035eb2a8c049ef787121476b" exitCode=0 Jul 12 00:16:16.500971 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:16.500630 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783380401072" event={"ID":"77477a44-d040-48d0-95da-bde461648995","Type":"ContainerDied","Data":"f865b407e4f61012c4e6598374f68b12ce9c8970035eb2a8c049ef787121476b"} Jul 12 00:16:17.636493 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.636455 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:17.668200 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.668175 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783380401072_77477a44-d040-48d0-95da-bde461648995/test-curl-7344-139783380401072/0.log" Jul 12 00:16:17.692274 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.692249 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783380401072"] Jul 12 00:16:17.695710 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.695685 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783380401072"] Jul 12 00:16:17.753720 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.753690 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g22jv\" (UniqueName: \"kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv\") pod \"77477a44-d040-48d0-95da-bde461648995\" (UID: \"77477a44-d040-48d0-95da-bde461648995\") " Jul 12 00:16:17.755876 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.755845 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv" (OuterVolumeSpecName: "kube-api-access-g22jv") pod "77477a44-d040-48d0-95da-bde461648995" (UID: "77477a44-d040-48d0-95da-bde461648995"). InnerVolumeSpecName "kube-api-access-g22jv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:17.818818 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.818792 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783381361840"] Jul 12 00:16:17.819159 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.819147 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77477a44-d040-48d0-95da-bde461648995" containerName="test-curl-7344-139783380401072" Jul 12 00:16:17.819202 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.819161 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="77477a44-d040-48d0-95da-bde461648995" containerName="test-curl-7344-139783380401072" Jul 12 00:16:17.819236 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.819224 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="77477a44-d040-48d0-95da-bde461648995" containerName="test-curl-7344-139783380401072" Jul 12 00:16:17.822163 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.822147 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:17.826011 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.825992 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381361840"] Jul 12 00:16:17.855202 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.855173 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g22jv\" (UniqueName: \"kubernetes.io/projected/77477a44-d040-48d0-95da-bde461648995-kube-api-access-g22jv\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:17.955967 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:17.955930 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zlfht\" (UniqueName: \"kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht\") pod \"test-curl-7344-139783381361840\" (UID: \"19450794-4dc7-4b54-b37b-1e2b7efc5028\") " pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:18.056833 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.056744 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zlfht\" (UniqueName: \"kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht\") pod \"test-curl-7344-139783381361840\" (UID: \"19450794-4dc7-4b54-b37b-1e2b7efc5028\") " pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:18.063654 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.063633 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zlfht\" (UniqueName: \"kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht\") pod \"test-curl-7344-139783381361840\" (UID: \"19450794-4dc7-4b54-b37b-1e2b7efc5028\") " pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:18.132501 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.132456 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:18.255910 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.255889 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381361840"] Jul 12 00:16:18.258018 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:18.257981 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod19450794_4dc7_4b54_b37b_1e2b7efc5028.slice/crio-455619c4b02232a8a36ca4b700f764d4b5390229be56d9b91d7611deda8bd23a WatchSource:0}: Error finding container 455619c4b02232a8a36ca4b700f764d4b5390229be56d9b91d7611deda8bd23a: Status 404 returned error can't find the container with id 455619c4b02232a8a36ca4b700f764d4b5390229be56d9b91d7611deda8bd23a Jul 12 00:16:18.515088 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.515049 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381361840" event={"ID":"19450794-4dc7-4b54-b37b-1e2b7efc5028","Type":"ContainerStarted","Data":"455619c4b02232a8a36ca4b700f764d4b5390229be56d9b91d7611deda8bd23a"} Jul 12 00:16:18.516313 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.516296 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="d26337c72266375c58d7f6b2230a7e012d47bb9c010ab8b9f912c3ec639f98a8" Jul 12 00:16:18.516423 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.516312 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:18.526235 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:16:18.526203 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7344-139783380401072\" is forbidden: User \"system:node:ip-10-0-140-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-223.ec2.internal' and this object" podUID="77477a44-d040-48d0-95da-bde461648995" pod="openshift-ingress/test-curl-7344-139783380401072" Jul 12 00:16:18.788811 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:18.788711 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77477a44-d040-48d0-95da-bde461648995" path="/var/lib/kubelet/pods/77477a44-d040-48d0-95da-bde461648995/volumes" Jul 12 00:16:19.523026 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:19.522992 2580 generic.go:356] "Generic (PLEG): container finished" podID="19450794-4dc7-4b54-b37b-1e2b7efc5028" containerID="bd653b03709b2323fd7daaf79cd319d081692679ce1c00b995dc992a687491e3" exitCode=0 Jul 12 00:16:19.523195 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:19.523067 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381361840" event={"ID":"19450794-4dc7-4b54-b37b-1e2b7efc5028","Type":"ContainerDied","Data":"bd653b03709b2323fd7daaf79cd319d081692679ce1c00b995dc992a687491e3"} Jul 12 00:16:19.715372 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:19.715343 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:16:20.672225 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.672202 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:20.689893 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.689872 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783381361840_19450794-4dc7-4b54-b37b-1e2b7efc5028/test-curl-7344-139783381361840/0.log" Jul 12 00:16:20.720686 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.720663 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783381361840"] Jul 12 00:16:20.722741 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.722720 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783381361840"] Jul 12 00:16:20.781178 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.781108 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zlfht\" (UniqueName: \"kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht\") pod \"19450794-4dc7-4b54-b37b-1e2b7efc5028\" (UID: \"19450794-4dc7-4b54-b37b-1e2b7efc5028\") " Jul 12 00:16:20.783195 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.783162 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht" (OuterVolumeSpecName: "kube-api-access-zlfht") pod "19450794-4dc7-4b54-b37b-1e2b7efc5028" (UID: "19450794-4dc7-4b54-b37b-1e2b7efc5028"). InnerVolumeSpecName "kube-api-access-zlfht". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:20.791639 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.791610 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="19450794-4dc7-4b54-b37b-1e2b7efc5028" path="/var/lib/kubelet/pods/19450794-4dc7-4b54-b37b-1e2b7efc5028/volumes" Jul 12 00:16:20.871916 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.871883 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:20.872259 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.872247 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="19450794-4dc7-4b54-b37b-1e2b7efc5028" containerName="test-curl-7344-139783381361840" Jul 12 00:16:20.872301 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.872261 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="19450794-4dc7-4b54-b37b-1e2b7efc5028" containerName="test-curl-7344-139783381361840" Jul 12 00:16:20.872343 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.872321 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="19450794-4dc7-4b54-b37b-1e2b7efc5028" containerName="test-curl-7344-139783381361840" Jul 12 00:16:20.875238 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.875222 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:20.879711 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.879684 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:20.882155 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.882131 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zlfht\" (UniqueName: \"kubernetes.io/projected/19450794-4dc7-4b54-b37b-1e2b7efc5028-kube-api-access-zlfht\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:20.980625 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.980595 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:16:20.982653 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.982628 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhg7c\" (UniqueName: \"kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c\") pod \"test-curl-7344-139783381442608\" (UID: \"c87df6ea-500e-4fbb-a751-1110103e6bfa\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:20.986690 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:20.986675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:16:21.084162 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.084069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zhg7c\" (UniqueName: \"kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c\") pod \"test-curl-7344-139783381442608\" (UID: \"c87df6ea-500e-4fbb-a751-1110103e6bfa\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:21.091454 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.091427 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhg7c\" (UniqueName: \"kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c\") pod \"test-curl-7344-139783381442608\" (UID: \"c87df6ea-500e-4fbb-a751-1110103e6bfa\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:21.185207 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.185175 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:21.311139 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.311114 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:21.312931 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:21.312904 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc87df6ea_500e_4fbb_a751_1110103e6bfa.slice/crio-28581e784a211c7775e1eb023c514c6bb495550d1000ccceefa2cd2f4837fadb WatchSource:0}: Error finding container 28581e784a211c7775e1eb023c514c6bb495550d1000ccceefa2cd2f4837fadb: Status 404 returned error can't find the container with id 28581e784a211c7775e1eb023c514c6bb495550d1000ccceefa2cd2f4837fadb Jul 12 00:16:21.533869 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.533836 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381442608" event={"ID":"c87df6ea-500e-4fbb-a751-1110103e6bfa","Type":"ContainerStarted","Data":"28581e784a211c7775e1eb023c514c6bb495550d1000ccceefa2cd2f4837fadb"} Jul 12 00:16:21.535087 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.535065 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381361840" Jul 12 00:16:21.535087 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:21.535065 2580 scope.go:117] "RemoveContainer" containerID="bd653b03709b2323fd7daaf79cd319d081692679ce1c00b995dc992a687491e3" Jul 12 00:16:22.541706 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:22.541671 2580 generic.go:356] "Generic (PLEG): container finished" podID="c87df6ea-500e-4fbb-a751-1110103e6bfa" containerID="f3ad5606f80e0e5a40680abf3156d2ec597be2e36ca2b38e65e2ea6de2779872" exitCode=0 Jul 12 00:16:22.542105 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:22.541750 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381442608" event={"ID":"c87df6ea-500e-4fbb-a751-1110103e6bfa","Type":"ContainerDied","Data":"f3ad5606f80e0e5a40680abf3156d2ec597be2e36ca2b38e65e2ea6de2779872"} Jul 12 00:16:23.675565 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.675540 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:23.694349 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.694313 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783381442608_c87df6ea-500e-4fbb-a751-1110103e6bfa/test-curl-7344-139783381442608/0.log" Jul 12 00:16:23.718089 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.718061 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:23.721836 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.721814 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:23.807773 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.807711 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhg7c\" (UniqueName: \"kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c\") pod \"c87df6ea-500e-4fbb-a751-1110103e6bfa\" (UID: \"c87df6ea-500e-4fbb-a751-1110103e6bfa\") " Jul 12 00:16:23.809670 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.809647 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c" (OuterVolumeSpecName: "kube-api-access-zhg7c") pod "c87df6ea-500e-4fbb-a751-1110103e6bfa" (UID: "c87df6ea-500e-4fbb-a751-1110103e6bfa"). InnerVolumeSpecName "kube-api-access-zhg7c". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:23.832946 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.832919 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:23.833324 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.833312 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c87df6ea-500e-4fbb-a751-1110103e6bfa" containerName="test-curl-7344-139783381442608" Jul 12 00:16:23.833367 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.833327 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c87df6ea-500e-4fbb-a751-1110103e6bfa" containerName="test-curl-7344-139783381442608" Jul 12 00:16:23.833422 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.833413 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c87df6ea-500e-4fbb-a751-1110103e6bfa" containerName="test-curl-7344-139783381442608" Jul 12 00:16:23.840273 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.838027 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:23.844524 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.844499 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:23.908804 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:23.908771 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhg7c\" (UniqueName: \"kubernetes.io/projected/c87df6ea-500e-4fbb-a751-1110103e6bfa-kube-api-access-zhg7c\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:24.010266 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.010208 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jwgsg\" (UniqueName: \"kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg\") pod \"test-curl-7344-139783381442608\" (UID: \"c6919c23-5c24-4b38-9451-193772e0956d\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:24.111448 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.111368 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jwgsg\" (UniqueName: \"kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg\") pod \"test-curl-7344-139783381442608\" (UID: \"c6919c23-5c24-4b38-9451-193772e0956d\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:24.118978 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.118954 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jwgsg\" (UniqueName: \"kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg\") pod \"test-curl-7344-139783381442608\" (UID: \"c6919c23-5c24-4b38-9451-193772e0956d\") " pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:24.149968 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.149942 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:24.276427 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.276395 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:24.277848 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:24.277822 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6919c23_5c24_4b38_9451_193772e0956d.slice/crio-4f4973826b8a13d1bfe8f1e1092be1aa1cd148ebb1a15f1bf4337f557730f882 WatchSource:0}: Error finding container 4f4973826b8a13d1bfe8f1e1092be1aa1cd148ebb1a15f1bf4337f557730f882: Status 404 returned error can't find the container with id 4f4973826b8a13d1bfe8f1e1092be1aa1cd148ebb1a15f1bf4337f557730f882 Jul 12 00:16:24.554447 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.554407 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381442608" event={"ID":"c6919c23-5c24-4b38-9451-193772e0956d","Type":"ContainerStarted","Data":"4f4973826b8a13d1bfe8f1e1092be1aa1cd148ebb1a15f1bf4337f557730f882"} Jul 12 00:16:24.555718 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.555700 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:24.555718 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.555713 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="28581e784a211c7775e1eb023c514c6bb495550d1000ccceefa2cd2f4837fadb" Jul 12 00:16:24.566971 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.566945 2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7344-139783381442608" oldPodUID="c87df6ea-500e-4fbb-a751-1110103e6bfa" podUID="c6919c23-5c24-4b38-9451-193772e0956d" Jul 12 00:16:24.788690 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:24.788650 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c87df6ea-500e-4fbb-a751-1110103e6bfa" path="/var/lib/kubelet/pods/c87df6ea-500e-4fbb-a751-1110103e6bfa/volumes" Jul 12 00:16:25.562908 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:25.562870 2580 generic.go:356] "Generic (PLEG): container finished" podID="c6919c23-5c24-4b38-9451-193772e0956d" containerID="a4878be19b5f903ef454d1d55265415324ef58bb9db272aa0ba864986a758ebf" exitCode=0 Jul 12 00:16:25.563085 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:25.562956 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381442608" event={"ID":"c6919c23-5c24-4b38-9451-193772e0956d","Type":"ContainerDied","Data":"a4878be19b5f903ef454d1d55265415324ef58bb9db272aa0ba864986a758ebf"} Jul 12 00:16:26.708486 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.708441 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:26.725462 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.725440 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783381442608_c6919c23-5c24-4b38-9451-193772e0956d/test-curl-7344-139783381442608/0.log" Jul 12 00:16:26.745979 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.745953 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:26.749595 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.749573 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783381442608"] Jul 12 00:16:26.833317 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.833243 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jwgsg\" (UniqueName: \"kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg\") pod \"c6919c23-5c24-4b38-9451-193772e0956d\" (UID: \"c6919c23-5c24-4b38-9451-193772e0956d\") " Jul 12 00:16:26.835246 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.835225 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg" (OuterVolumeSpecName: "kube-api-access-jwgsg") pod "c6919c23-5c24-4b38-9451-193772e0956d" (UID: "c6919c23-5c24-4b38-9451-193772e0956d"). InnerVolumeSpecName "kube-api-access-jwgsg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:26.868392 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.868362 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:26.868730 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.868717 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c6919c23-5c24-4b38-9451-193772e0956d" containerName="test-curl-7344-139783381442608" Jul 12 00:16:26.868778 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.868732 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c6919c23-5c24-4b38-9451-193772e0956d" containerName="test-curl-7344-139783381442608" Jul 12 00:16:26.868821 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.868812 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c6919c23-5c24-4b38-9451-193772e0956d" containerName="test-curl-7344-139783381442608" Jul 12 00:16:26.871698 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.871680 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:26.874647 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.874629 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:26.934625 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:26.934593 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jwgsg\" (UniqueName: \"kubernetes.io/projected/c6919c23-5c24-4b38-9451-193772e0956d-kube-api-access-jwgsg\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:27.035209 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.035171 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n8q5w\" (UniqueName: \"kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w\") pod \"test-curl-7344-139783381443616\" (UID: \"7aabbcd4-ea67-45d4-8400-2a36827f223c\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:27.135790 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.135701 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-n8q5w\" (UniqueName: \"kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w\") pod \"test-curl-7344-139783381443616\" (UID: \"7aabbcd4-ea67-45d4-8400-2a36827f223c\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:27.142875 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.142849 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-n8q5w\" (UniqueName: \"kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w\") pod \"test-curl-7344-139783381443616\" (UID: \"7aabbcd4-ea67-45d4-8400-2a36827f223c\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:27.182437 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.182404 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:27.308733 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.308706 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:27.310328 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:27.310303 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7aabbcd4_ea67_45d4_8400_2a36827f223c.slice/crio-bab1cb6f3d0afdac89ab09ca970cb183c1dd2e9ead620c1eb4ccb056a4a227be WatchSource:0}: Error finding container bab1cb6f3d0afdac89ab09ca970cb183c1dd2e9ead620c1eb4ccb056a4a227be: Status 404 returned error can't find the container with id bab1cb6f3d0afdac89ab09ca970cb183c1dd2e9ead620c1eb4ccb056a4a227be Jul 12 00:16:27.574500 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.574451 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381443616" event={"ID":"7aabbcd4-ea67-45d4-8400-2a36827f223c","Type":"ContainerStarted","Data":"bab1cb6f3d0afdac89ab09ca970cb183c1dd2e9ead620c1eb4ccb056a4a227be"} Jul 12 00:16:27.575558 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.575540 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381442608" Jul 12 00:16:27.575680 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:27.575583 2580 scope.go:117] "RemoveContainer" containerID="a4878be19b5f903ef454d1d55265415324ef58bb9db272aa0ba864986a758ebf" Jul 12 00:16:28.587395 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:28.587360 2580 generic.go:356] "Generic (PLEG): container finished" podID="7aabbcd4-ea67-45d4-8400-2a36827f223c" containerID="616cd8f32aec933fc1f0be2b8244e6977db97812597ab1a31727976c25b44c5d" exitCode=0 Jul 12 00:16:28.587787 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:28.587421 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381443616" event={"ID":"7aabbcd4-ea67-45d4-8400-2a36827f223c","Type":"ContainerDied","Data":"616cd8f32aec933fc1f0be2b8244e6977db97812597ab1a31727976c25b44c5d"} Jul 12 00:16:28.788885 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:28.788854 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c6919c23-5c24-4b38-9451-193772e0956d" path="/var/lib/kubelet/pods/c6919c23-5c24-4b38-9451-193772e0956d/volumes" Jul 12 00:16:29.732158 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.732130 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:29.752706 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.752680 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783381443616_7aabbcd4-ea67-45d4-8400-2a36827f223c/test-curl-7344-139783381443616/0.log" Jul 12 00:16:29.754938 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.754913 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n8q5w\" (UniqueName: \"kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w\") pod \"7aabbcd4-ea67-45d4-8400-2a36827f223c\" (UID: \"7aabbcd4-ea67-45d4-8400-2a36827f223c\") " Jul 12 00:16:29.756851 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.756823 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w" (OuterVolumeSpecName: "kube-api-access-n8q5w") pod "7aabbcd4-ea67-45d4-8400-2a36827f223c" (UID: "7aabbcd4-ea67-45d4-8400-2a36827f223c"). InnerVolumeSpecName "kube-api-access-n8q5w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:29.776259 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.776238 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:29.779904 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.779880 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:29.855972 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.855906 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n8q5w\" (UniqueName: \"kubernetes.io/projected/7aabbcd4-ea67-45d4-8400-2a36827f223c-kube-api-access-n8q5w\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:29.905322 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.905292 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:29.905680 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.905667 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7aabbcd4-ea67-45d4-8400-2a36827f223c" containerName="test-curl-7344-139783381443616" Jul 12 00:16:29.905732 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.905682 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="7aabbcd4-ea67-45d4-8400-2a36827f223c" containerName="test-curl-7344-139783381443616" Jul 12 00:16:29.905770 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.905742 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="7aabbcd4-ea67-45d4-8400-2a36827f223c" containerName="test-curl-7344-139783381443616" Jul 12 00:16:29.908105 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.908090 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:29.915337 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.915313 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:29.956852 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:29.956828 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bx72k\" (UniqueName: \"kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k\") pod \"test-curl-7344-139783381443616\" (UID: \"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:30.057802 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.057768 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bx72k\" (UniqueName: \"kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k\") pod \"test-curl-7344-139783381443616\" (UID: \"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:30.065052 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.065025 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bx72k\" (UniqueName: \"kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k\") pod \"test-curl-7344-139783381443616\" (UID: \"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49\") " pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:30.218170 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.218080 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:30.345134 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.345110 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:30.347108 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:16:30.347079 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podebc1e63b_dcb9_47f1_93cf_0218f9b53a49.slice/crio-e4d04e49ab8c4df35cd07807107dfa6226c33edf579595a605ea388a905059cb WatchSource:0}: Error finding container e4d04e49ab8c4df35cd07807107dfa6226c33edf579595a605ea388a905059cb: Status 404 returned error can't find the container with id e4d04e49ab8c4df35cd07807107dfa6226c33edf579595a605ea388a905059cb Jul 12 00:16:30.599729 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.599702 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:30.599729 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.599727 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="bab1cb6f3d0afdac89ab09ca970cb183c1dd2e9ead620c1eb4ccb056a4a227be" Jul 12 00:16:30.600823 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.600798 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381443616" event={"ID":"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49","Type":"ContainerStarted","Data":"e4d04e49ab8c4df35cd07807107dfa6226c33edf579595a605ea388a905059cb"} Jul 12 00:16:30.610784 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.610761 2580 status_manager.go:1027] "Pod was deleted and then recreated, skipping status update" pod="openshift-ingress/test-curl-7344-139783381443616" oldPodUID="7aabbcd4-ea67-45d4-8400-2a36827f223c" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" Jul 12 00:16:30.788626 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:30.788598 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7aabbcd4-ea67-45d4-8400-2a36827f223c" path="/var/lib/kubelet/pods/7aabbcd4-ea67-45d4-8400-2a36827f223c/volumes" Jul 12 00:16:31.608253 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:31.608219 2580 generic.go:356] "Generic (PLEG): container finished" podID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" containerID="ca494eb53053ad07340805c5e4c1a442881790e159561e30e5d72180d704f93c" exitCode=0 Jul 12 00:16:31.608433 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:31.608262 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/test-curl-7344-139783381443616" event={"ID":"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49","Type":"ContainerDied","Data":"ca494eb53053ad07340805c5e4c1a442881790e159561e30e5d72180d704f93c"} Jul 12 00:16:32.751768 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.751740 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:32.770502 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.770458 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_test-curl-7344-139783381443616_ebc1e63b-dcb9-47f1-93cf-0218f9b53a49/test-curl-7344-139783381443616/0.log" Jul 12 00:16:32.782755 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.782734 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bx72k\" (UniqueName: \"kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k\") pod \"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49\" (UID: \"ebc1e63b-dcb9-47f1-93cf-0218f9b53a49\") " Jul 12 00:16:32.784786 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.784759 2580 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k" (OuterVolumeSpecName: "kube-api-access-bx72k") pod "ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" (UID: "ebc1e63b-dcb9-47f1-93cf-0218f9b53a49"). InnerVolumeSpecName "kube-api-access-bx72k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 12 00:16:32.793542 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.793521 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:32.799589 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.799570 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/test-curl-7344-139783381443616"] Jul 12 00:16:32.883564 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:32.883455 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bx72k\" (UniqueName: \"kubernetes.io/projected/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49-kube-api-access-bx72k\") on node \"ip-10-0-140-223.ec2.internal\" DevicePath \"\"" Jul 12 00:16:33.620568 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:33.620537 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:33.620739 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:33.620580 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e4d04e49ab8c4df35cd07807107dfa6226c33edf579595a605ea388a905059cb" Jul 12 00:16:33.624844 ip-10-0-140-223 kubenswrapper[2580]: E0712 00:16:33.624811 2580 status_manager.go:1018] "Failed to get status for pod" err="pods \"test-curl-7344-139783381443616\" is forbidden: User \"system:node:ip-10-0-140-223.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-ingress\": no relationship found between node 'ip-10-0-140-223.ec2.internal' and this object" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" pod="openshift-ingress/test-curl-7344-139783381443616" Jul 12 00:16:34.788091 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:34.788064 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" path="/var/lib/kubelet/pods/ebc1e63b-dcb9-47f1-93cf-0218f9b53a49/volumes" Jul 12 00:16:47.015157 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:16:47.015117 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:17:12.621718 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:12.621683 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-5gfdw"] Jul 12 00:17:25.526897 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:25.526850 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b954d4f86-w5nbl_2ba8e080-f88f-4de4-a251-fe177e308638/authorino/0.log" Jul 12 00:17:31.269062 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:31.269035 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-4jvrg_10e36855-88f4-491a-8b60-b9feacc5b23f/copy-manifests/0.log" Jul 12 00:17:31.276567 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:31.276536 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-4jvrg_10e36855-88f4-491a-8b60-b9feacc5b23f/manager/0.log" Jul 12 00:17:32.603060 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:32.603028 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/odh-ai-gateway-infra_maas-api-revoke-keys-e2e-api-a-665549-cvnjn_ebabcd0e-09a2-4208-9a67-f16b010a2e99/revoke-keys/0.log" Jul 12 00:17:34.563207 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.563176 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/util/0.log" Jul 12 00:17:34.569064 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.569040 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/pull/0.log" Jul 12 00:17:34.574586 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.574565 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/extract/0.log" Jul 12 00:17:34.686070 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.686039 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/pull/0.log" Jul 12 00:17:34.691935 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.691908 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/extract/0.log" Jul 12 00:17:34.697632 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.697611 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/util/0.log" Jul 12 00:17:34.820464 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.820383 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/util/0.log" Jul 12 00:17:34.826703 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.826679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/pull/0.log" Jul 12 00:17:34.832962 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.832943 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/extract/0.log" Jul 12 00:17:34.949943 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.949915 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/util/0.log" Jul 12 00:17:34.956694 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.956673 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/pull/0.log" Jul 12 00:17:34.963748 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:34.963727 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/extract/0.log" Jul 12 00:17:35.085135 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:35.085058 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b954d4f86-w5nbl_2ba8e080-f88f-4de4-a251-fe177e308638/authorino/0.log" Jul 12 00:17:35.444033 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:35.443955 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kj6d4_c53eb2ab-ed74-4846-8d11-5d547275a91c/kuadrant-console-plugin/0.log" Jul 12 00:17:35.837728 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:35.837698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5gfdw_0c131ed1-3aee-4a03-8ee2-3005cfd1d46b/limitador/0.log" Jul 12 00:17:36.658496 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:36.658444 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-nmwmf_7357cf84-807d-42d6-8f25-c558a74dd56f/kube-auth-proxy/0.log" Jul 12 00:17:37.843334 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:37.843299 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbtp8fh_4158fcb0-42c5-49da-bc22-0e554c16d5fb/main/0.log" Jul 12 00:17:42.210166 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.210077 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-scjg4/must-gather-bz7rz"] Jul 12 00:17:42.210633 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.210515 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" containerName="test-curl-7344-139783381443616" Jul 12 00:17:42.210633 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.210530 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" containerName="test-curl-7344-139783381443616" Jul 12 00:17:42.210633 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.210612 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="ebc1e63b-dcb9-47f1-93cf-0218f9b53a49" containerName="test-curl-7344-139783381443616" Jul 12 00:17:42.214999 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.214973 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.219270 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.219244 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scjg4\"/\"kube-root-ca.crt\"" Jul 12 00:17:42.220557 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.220454 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-scjg4\"/\"openshift-service-ca.crt\"" Jul 12 00:17:42.222381 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.222361 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-scjg4\"/\"default-dockercfg-hhxcj\"" Jul 12 00:17:42.222634 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.222615 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/must-gather-bz7rz"] Jul 12 00:17:42.305798 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.305761 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4eeca63f-6c92-4d2f-8c48-2a63f271269e-must-gather-output\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.305991 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.305826 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-td8h4\" (UniqueName: \"kubernetes.io/projected/4eeca63f-6c92-4d2f-8c48-2a63f271269e-kube-api-access-td8h4\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.406927 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.406886 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-td8h4\" (UniqueName: \"kubernetes.io/projected/4eeca63f-6c92-4d2f-8c48-2a63f271269e-kube-api-access-td8h4\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.407107 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.406967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4eeca63f-6c92-4d2f-8c48-2a63f271269e-must-gather-output\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.407297 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.407283 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/4eeca63f-6c92-4d2f-8c48-2a63f271269e-must-gather-output\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.414653 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.414625 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-td8h4\" (UniqueName: \"kubernetes.io/projected/4eeca63f-6c92-4d2f-8c48-2a63f271269e-kube-api-access-td8h4\") pod \"must-gather-bz7rz\" (UID: \"4eeca63f-6c92-4d2f-8c48-2a63f271269e\") " pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.527846 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.527814 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/must-gather-bz7rz" Jul 12 00:17:42.662511 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:42.662456 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/must-gather-bz7rz"] Jul 12 00:17:42.664130 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:17:42.664100 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4eeca63f_6c92_4d2f_8c48_2a63f271269e.slice/crio-d5e30f4cf4ba0a1fa056e0eb12a7054830167b9d7950acdebe2c017dc0c49b50 WatchSource:0}: Error finding container d5e30f4cf4ba0a1fa056e0eb12a7054830167b9d7950acdebe2c017dc0c49b50: Status 404 returned error can't find the container with id d5e30f4cf4ba0a1fa056e0eb12a7054830167b9d7950acdebe2c017dc0c49b50 Jul 12 00:17:43.027201 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:43.027164 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/must-gather-bz7rz" event={"ID":"4eeca63f-6c92-4d2f-8c48-2a63f271269e","Type":"ContainerStarted","Data":"d5e30f4cf4ba0a1fa056e0eb12a7054830167b9d7950acdebe2c017dc0c49b50"} Jul 12 00:17:44.035958 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:44.035924 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/must-gather-bz7rz" event={"ID":"4eeca63f-6c92-4d2f-8c48-2a63f271269e","Type":"ContainerStarted","Data":"8d961d2815c8b8f84584aa9ade5be065708bc53c897cabaf588ebb04a50ced10"} Jul 12 00:17:44.036335 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:44.035970 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/must-gather-bz7rz" event={"ID":"4eeca63f-6c92-4d2f-8c48-2a63f271269e","Type":"ContainerStarted","Data":"8e5cce0c27194894b873903c937001c236dbdac97283626230789a77f4f4cd93"} Jul 12 00:17:44.052636 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:44.052588 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-scjg4/must-gather-bz7rz" podStartSLOduration=1.145906057 podStartE2EDuration="2.052572489s" totalImagesPullingTime="906.666432ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-12 00:17:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:17:44.049770213 +0000 UTC m=+2783.801517740" watchObservedRunningTime="2026-07-12 00:17:44.052572489 +0000 UTC m=+2783.804319991" Jul 12 00:17:45.097889 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:45.097848 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gstql_f5de7bc3-7a27-4033-9303-380b33e254f9/global-pull-secret-syncer/0.log" Jul 12 00:17:45.206761 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:45.206732 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-rw7vw_886f3bd4-de43-4d6c-a07c-0b3b3c76a0ab/konnectivity-agent/0.log" Jul 12 00:17:45.293884 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:45.293842 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-140-223.ec2.internal_16a2c0a00cdbe9d77d0bff3f91f1c3e7/haproxy/0.log" Jul 12 00:17:49.497387 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.497354 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/extract/0.log" Jul 12 00:17:49.524214 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.524183 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/util/0.log" Jul 12 00:17:49.547607 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.547580 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759c9ffd_c3a1f993-3bb8-4655-831d-6621fd9a1078/pull/0.log" Jul 12 00:17:49.576012 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.575983 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/extract/0.log" Jul 12 00:17:49.608911 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.608875 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/util/0.log" Jul 12 00:17:49.635233 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.635201 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0tp7gz_7093ffb8-9b32-45c0-ac31-9bef83bca313/pull/0.log" Jul 12 00:17:49.677954 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.677920 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/extract/0.log" Jul 12 00:17:49.704146 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.704106 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/util/0.log" Jul 12 00:17:49.725046 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.725018 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed737drtr_4bdbd3ad-5c9b-42fa-8722-17f9a1f86dda/pull/0.log" Jul 12 00:17:49.755674 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.755550 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/extract/0.log" Jul 12 00:17:49.780154 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.780049 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/util/0.log" Jul 12 00:17:49.799752 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:49.799714 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1wn57c_10eaa2bc-ab69-49a9-a1e4-1baf05dfc8a0/pull/0.log" Jul 12 00:17:50.209761 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:50.209665 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-7b954d4f86-w5nbl_2ba8e080-f88f-4de4-a251-fe177e308638/authorino/0.log" Jul 12 00:17:50.287335 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:50.287293 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-kj6d4_c53eb2ab-ed74-4846-8d11-5d547275a91c/kuadrant-console-plugin/0.log" Jul 12 00:17:50.490590 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:50.490564 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-5gfdw_0c131ed1-3aee-4a03-8ee2-3005cfd1d46b/limitador/0.log" Jul 12 00:17:52.222882 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.222818 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/alertmanager/0.log" Jul 12 00:17:52.246424 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.246371 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/config-reloader/0.log" Jul 12 00:17:52.269374 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.269347 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/kube-rbac-proxy-web/0.log" Jul 12 00:17:52.290428 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.290402 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/kube-rbac-proxy/0.log" Jul 12 00:17:52.312580 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.312555 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/kube-rbac-proxy-metric/0.log" Jul 12 00:17:52.335529 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.335501 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/prom-label-proxy/0.log" Jul 12 00:17:52.357488 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.357445 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_836cfc2e-0604-4065-bd43-6baf81cf90dd/init-config-reloader/0.log" Jul 12 00:17:52.440452 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.440416 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wlqh4_e008a97f-3dfd-4ca0-b4fb-14717a72f1b8/kube-state-metrics/0.log" Jul 12 00:17:52.461951 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.461908 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wlqh4_e008a97f-3dfd-4ca0-b4fb-14717a72f1b8/kube-rbac-proxy-main/0.log" Jul 12 00:17:52.485312 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.485199 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-785778ff74-wlqh4_e008a97f-3dfd-4ca0-b4fb-14717a72f1b8/kube-rbac-proxy-self/0.log" Jul 12 00:17:52.515024 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.514990 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6798bd576f-4dg96_1374e254-a8cc-47cc-99dd-414a8bb6b7ff/metrics-server/0.log" Jul 12 00:17:52.543520 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.543443 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-mzr2j_248746d9-aed0-4297-81ad-c44dc684a255/monitoring-plugin/0.log" Jul 12 00:17:52.683222 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.683167 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qt64h_a0fc3079-83c6-4d06-bed2-d73eb17109fa/node-exporter/0.log" Jul 12 00:17:52.707125 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.707099 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qt64h_a0fc3079-83c6-4d06-bed2-d73eb17109fa/kube-rbac-proxy/0.log" Jul 12 00:17:52.737874 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.737801 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-qt64h_a0fc3079-83c6-4d06-bed2-d73eb17109fa/init-textfile/0.log" Jul 12 00:17:52.860121 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.860094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lc77j_02312fb7-9694-4563-b92f-f2584c72ef7e/kube-rbac-proxy-main/0.log" Jul 12 00:17:52.888786 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.888291 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lc77j_02312fb7-9694-4563-b92f-f2584c72ef7e/kube-rbac-proxy-self/0.log" Jul 12 00:17:52.921594 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:52.921564 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-lc77j_02312fb7-9694-4563-b92f-f2584c72ef7e/openshift-state-metrics/0.log" Jul 12 00:17:53.151187 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.151094 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-4qgln_e0f41fe7-4cd6-496a-b762-132e803f1af7/prometheus-operator/0.log" Jul 12 00:17:53.173588 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.173557 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-64785b6675-4qgln_e0f41fe7-4cd6-496a-b762-132e803f1af7/kube-rbac-proxy/0.log" Jul 12 00:17:53.199771 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.199737 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-gtn6q_ed966a69-ddc7-4d00-818b-fe13932813dc/prometheus-operator-admission-webhook/0.log" Jul 12 00:17:53.258742 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.258703 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fcbdffc7-v4kdr_7cbaa007-2345-40a1-8734-1e5cf935534d/telemeter-client/0.log" Jul 12 00:17:53.280697 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.280664 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fcbdffc7-v4kdr_7cbaa007-2345-40a1-8734-1e5cf935534d/reload/0.log" Jul 12 00:17:53.301931 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.301886 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7fcbdffc7-v4kdr_7cbaa007-2345-40a1-8734-1e5cf935534d/kube-rbac-proxy/0.log" Jul 12 00:17:53.332723 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.332691 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/thanos-query/0.log" Jul 12 00:17:53.353535 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.353503 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/kube-rbac-proxy-web/0.log" Jul 12 00:17:53.374322 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.374295 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/kube-rbac-proxy/0.log" Jul 12 00:17:53.394926 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.394892 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/prom-label-proxy/0.log" Jul 12 00:17:53.417599 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.417420 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/kube-rbac-proxy-rules/0.log" Jul 12 00:17:53.441237 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:53.441208 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-668c9bc467-7kczj_76f110e1-a005-49f5-ad46-ea8831fd7440/kube-rbac-proxy-metrics/0.log" Jul 12 00:17:55.515125 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.515082 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv"] Jul 12 00:17:55.520658 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.520625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.528567 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.528530 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv"] Jul 12 00:17:55.653023 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.652975 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-proc\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.653215 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.653045 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-sys\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.653215 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.653161 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6cz68\" (UniqueName: \"kubernetes.io/projected/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-kube-api-access-6cz68\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.653404 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.653229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-lib-modules\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.653404 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.653254 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-podres\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.754600 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.754557 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6cz68\" (UniqueName: \"kubernetes.io/projected/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-kube-api-access-6cz68\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.754922 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.754899 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-lib-modules\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.755058 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.755042 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-podres\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.755216 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.755192 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-proc\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.755346 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.755331 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-sys\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.755605 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.755588 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-sys\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.756186 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.756163 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-lib-modules\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.756420 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.756403 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-podres\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.756603 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.756581 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-proc\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.763665 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.763628 2580 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6cz68\" (UniqueName: \"kubernetes.io/projected/54325e73-1c56-4bb4-b7bc-2d73eadb6e5d-kube-api-access-6cz68\") pod \"perf-node-gather-daemonset-zqpqv\" (UID: \"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d\") " pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:55.831922 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.831839 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-786d79c5b5-s4tzk_65c156a7-94da-45f3-a438-634545760468/console/0.log" Jul 12 00:17:55.835700 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:55.835674 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:56.042679 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:56.042624 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv"] Jul 12 00:17:56.052956 ip-10-0-140-223 kubenswrapper[2580]: W0712 00:17:56.050803 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod54325e73_1c56_4bb4_b7bc_2d73eadb6e5d.slice/crio-5d75c1bcac5209ee7fbacee259b3d5c9bee1bcfbac608e78f0997fbcef20e030 WatchSource:0}: Error finding container 5d75c1bcac5209ee7fbacee259b3d5c9bee1bcfbac608e78f0997fbcef20e030: Status 404 returned error can't find the container with id 5d75c1bcac5209ee7fbacee259b3d5c9bee1bcfbac608e78f0997fbcef20e030 Jul 12 00:17:56.139205 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:56.139166 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" event={"ID":"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d","Type":"ContainerStarted","Data":"5d75c1bcac5209ee7fbacee259b3d5c9bee1bcfbac608e78f0997fbcef20e030"} Jul 12 00:17:57.151168 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.151024 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" event={"ID":"54325e73-1c56-4bb4-b7bc-2d73eadb6e5d","Type":"ContainerStarted","Data":"b32a7b77398b159c3c1851f524c07a85b084afca1f6ca2963b2cd0d0be92b06d"} Jul 12 00:17:57.151648 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.151450 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:17:57.169199 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.169126 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" podStartSLOduration=2.169100858 podStartE2EDuration="2.169100858s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-12 00:17:55 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-12 00:17:57.165934181 +0000 UTC m=+2796.917681689" watchObservedRunningTime="2026-07-12 00:17:57.169100858 +0000 UTC m=+2796.920848362" Jul 12 00:17:57.598631 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.598585 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qxk48_f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b/dns/0.log" Jul 12 00:17:57.620169 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.620131 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-qxk48_f2dec4fb-b5f0-408a-aad5-b94ec6b85e5b/kube-rbac-proxy/0.log" Jul 12 00:17:57.642870 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:57.642836 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-2qvz4_7b085832-ada8-407e-8fb8-ab9a7c310ee9/dns-node-resolver/0.log" Jul 12 00:17:58.205970 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:58.205944 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-pruner-29730240-v24zp_08260403-1024-4a91-8c28-ac844874192b/image-pruner/0.log" Jul 12 00:17:58.288086 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:58.288055 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-m92gl_c3208457-f67d-4e7d-bfe6-b57b1baa6d7b/node-ca/0.log" Jul 12 00:17:59.403222 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:17:59.403190 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-nmwmf_7357cf84-807d-42d6-8f25-c558a74dd56f/kube-auth-proxy/0.log" Jul 12 00:18:00.282907 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:00.282876 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-ngdqs_9529b946-8c65-47e9-90a3-1f33d9188cf1/serve-healthcheck-canary/0.log" Jul 12 00:18:00.886717 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:00.886675 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vt9sd_9d8c2819-ee75-40b5-ade5-328096d0c06a/kube-rbac-proxy/0.log" Jul 12 00:18:00.908878 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:00.908854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vt9sd_9d8c2819-ee75-40b5-ade5-328096d0c06a/exporter/0.log" Jul 12 00:18:00.930737 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:00.930712 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-vt9sd_9d8c2819-ee75-40b5-ade5-328096d0c06a/extractor/0.log" Jul 12 00:18:03.175006 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:03.174973 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-scjg4/perf-node-gather-daemonset-zqpqv" Jul 12 00:18:03.328727 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:03.328698 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-4jvrg_10e36855-88f4-491a-8b60-b9feacc5b23f/manager/0.log" Jul 12 00:18:03.347924 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:03.347898 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-6d55578dd8-4jvrg_10e36855-88f4-491a-8b60-b9feacc5b23f/copy-manifests/0.log" Jul 12 00:18:04.821164 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:04.821135 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-59d758c946-4ljpl_b47a2fef-49dd-45b8-80ec-2b6a175ae81c/manager/0.log" Jul 12 00:18:04.876091 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:04.876056 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-sx72k_5eca7cd1-b772-4025-a4b9-8b01fb68a490/openshift-lws-operator/0.log" Jul 12 00:18:05.848025 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:05.847997 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-nmwmf_7357cf84-807d-42d6-8f25-c558a74dd56f/kube-auth-proxy/0.log" Jul 12 00:18:07.296380 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:07.296353 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7bd58cf797-nmwmf_7357cf84-807d-42d6-8f25-c558a74dd56f/kube-auth-proxy/0.log" Jul 12 00:18:11.230120 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.230080 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-88pmr_feca8d66-b572-4fca-bcc0-2eda7e4a1d2b/kube-multus/0.log" Jul 12 00:18:11.548521 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.548431 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/kube-multus-additional-cni-plugins/0.log" Jul 12 00:18:11.571633 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.571605 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/egress-router-binary-copy/0.log" Jul 12 00:18:11.605530 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.605497 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/cni-plugins/0.log" Jul 12 00:18:11.625479 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.625436 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/bond-cni-plugin/0.log" Jul 12 00:18:11.651038 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.651017 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/routeoverride-cni/0.log" Jul 12 00:18:11.670798 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.670773 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/whereabouts-cni-bincopy/0.log" Jul 12 00:18:11.690046 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.690023 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-mbh6w_77b79610-2e35-4d57-bca6-5ac0ee3d0b2f/whereabouts-cni/0.log" Jul 12 00:18:11.864729 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.864647 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cdt7s_338bdff5-9113-462b-a1db-947024a1db1d/network-metrics-daemon/0.log" Jul 12 00:18:11.883516 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:11.883464 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cdt7s_338bdff5-9113-462b-a1db-947024a1db1d/kube-rbac-proxy/0.log" Jul 12 00:18:12.682849 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.682772 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-controller/0.log" Jul 12 00:18:12.699733 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.699701 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/1.log" Jul 12 00:18:12.699913 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.699825 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovn-acl-logging/0.log" Jul 12 00:18:12.717372 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.717339 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/kube-rbac-proxy-node/0.log" Jul 12 00:18:12.741382 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.741320 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/kube-rbac-proxy-ovn-metrics/0.log" Jul 12 00:18:12.763017 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.762985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/northd/0.log" Jul 12 00:18:12.782954 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.782922 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/nbdb/0.log" Jul 12 00:18:12.802444 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.802421 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/sbdb/0.log" Jul 12 00:18:12.870320 ip-10-0-140-223 kubenswrapper[2580]: I0712 00:18:12.870294 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-dwvq7_5edd8b3d-0380-41e5-b734-655dd000e945/ovnkube-controller/0.log"