Jul 08 17:17:24.085769 ip-10-0-141-16 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 08 17:17:24.085782 ip-10-0-141-16 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 08 17:17:24.085791 ip-10-0-141-16 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 08 17:17:24.086022 ip-10-0-141-16 systemd[1]: Failed to start Kubernetes Kubelet. Jul 08 17:17:34.236252 ip-10-0-141-16 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 08 17:17:34.236267 ip-10-0-141-16 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 337be2a96e514621a9e58402ccd05d54 -- Jul 08 17:19:26.943779 ip-10-0-141-16 systemd[1]: Starting Kubernetes Kubelet... Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 08 17:19:27.474388 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.473850 2551 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479323 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479341 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479348 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479352 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479356 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479361 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479364 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479367 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479373 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479377 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479380 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479386 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479391 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479397 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479401 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479404 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479409 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479418 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.480129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479421 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479425 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479428 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479431 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479434 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479439 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479441 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479444 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479448 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479453 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479455 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479458 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479461 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479466 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479469 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479472 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479477 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479480 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479482 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479485 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.481067 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479490 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479493 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479496 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479499 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479505 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479507 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479510 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479515 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479519 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479521 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479527 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479529 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479532 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479535 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479540 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479543 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479546 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479548 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479565 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479568 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.482471 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479571 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479574 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479580 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479582 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479590 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479594 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479597 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479602 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479605 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479608 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479611 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479616 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479620 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479628 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479631 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479636 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479640 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479643 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479646 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.483421 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479649 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479652 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479655 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479657 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479661 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479665 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479668 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479671 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479673 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479678 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479681 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479684 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479687 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479692 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479695 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479698 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479704 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479707 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479709 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479712 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.484371 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479717 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.479720 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481582 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481594 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481600 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481605 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481611 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481615 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481619 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481624 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481628 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481631 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481636 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481639 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481642 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481647 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481650 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481654 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481659 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481662 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.485836 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481665 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481668 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481674 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481676 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481679 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481682 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481687 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481691 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481697 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481700 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481705 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481712 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481715 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481718 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481723 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481726 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481729 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481732 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481737 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.486806 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481740 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481744 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481749 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481752 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481754 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481757 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481760 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481763 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481766 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481769 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481771 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481774 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481779 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481782 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481785 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481790 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481793 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481796 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481798 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481803 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.488014 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481806 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481809 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481812 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481816 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481819 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481822 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481824 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481829 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481832 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481835 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481838 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481840 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481846 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481848 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481851 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481858 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481861 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481863 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481868 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481871 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.488948 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481874 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481877 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481881 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481884 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481888 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481893 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481896 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481899 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481901 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481906 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481909 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481912 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481914 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481919 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481922 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481925 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481930 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481933 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481936 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481942 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.490129 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481945 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.481947 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482435 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482440 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482443 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482446 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482449 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482452 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482455 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482458 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482461 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482465 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482468 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482471 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482474 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482481 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482486 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482491 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482495 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482498 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.491074 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482502 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482506 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482508 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482512 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482516 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482519 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482522 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482530 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482533 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482535 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482538 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482543 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482546 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482563 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482570 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482573 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482575 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482581 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482584 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482586 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.492479 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482589 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482592 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482597 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482600 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482604 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482608 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482614 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482617 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482620 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482626 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482629 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482631 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482634 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482639 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482642 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482645 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482650 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482653 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482656 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.493412 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482664 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482668 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482673 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482676 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482679 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482681 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482684 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482688 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482691 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482694 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482697 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482700 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482702 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482707 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482710 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482712 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482717 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482720 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482724 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482726 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.494825 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482731 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482734 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482737 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482741 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482745 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482748 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482751 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482754 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482758 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482761 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482764 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482769 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482774 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482776 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482781 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482784 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482787 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482790 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482798 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482802 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.495989 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482805 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.482808 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482907 2551 flags.go:64] FLAG: --address="0.0.0.0" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482916 2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482923 2551 flags.go:64] FLAG: --anonymous-auth="true" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482928 2551 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482932 2551 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482936 2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482942 2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482947 2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482950 2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482956 2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482961 2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482966 2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482970 2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482973 2551 flags.go:64] FLAG: --cgroup-root="" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482978 2551 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482983 2551 flags.go:64] FLAG: --client-ca-file="" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482987 2551 flags.go:64] FLAG: --cloud-provider="external" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482990 2551 flags.go:64] FLAG: --cluster-dns="[]" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.482997 2551 flags.go:64] FLAG: --cluster-domain="" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483000 2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483004 2551 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483009 2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 08 17:19:27.497161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483015 2551 flags.go:64] FLAG: --container-log-max-files="5" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483019 2551 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483025 2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483029 2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483032 2551 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483038 2551 flags.go:64] FLAG: --contention-profiling="false" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483043 2551 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483048 2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483051 2551 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483055 2551 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483061 2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483065 2551 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483068 2551 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483073 2551 flags.go:64] FLAG: --enable-load-reader="false" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483077 2551 flags.go:64] FLAG: --enable-server="true" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483080 2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483086 2551 flags.go:64] FLAG: --event-burst="100" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483090 2551 flags.go:64] FLAG: --event-qps="50" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483093 2551 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483096 2551 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483101 2551 flags.go:64] FLAG: --eviction-hard="" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483105 2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483108 2551 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483113 2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483117 2551 flags.go:64] FLAG: --eviction-soft="" Jul 08 17:19:27.498357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483121 2551 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483126 2551 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483129 2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483133 2551 flags.go:64] FLAG: --experimental-mounter-path="" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483136 2551 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483141 2551 flags.go:64] FLAG: --fail-swap-on="true" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483144 2551 flags.go:64] FLAG: --feature-gates="" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483152 2551 flags.go:64] FLAG: --file-check-frequency="20s" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483155 2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483158 2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483162 2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483165 2551 flags.go:64] FLAG: --healthz-port="10248" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483168 2551 flags.go:64] FLAG: --help="false" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483173 2551 flags.go:64] FLAG: --hostname-override="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483176 2551 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483181 2551 flags.go:64] FLAG: --http-check-frequency="20s" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483184 2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483187 2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483191 2551 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483196 2551 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483199 2551 flags.go:64] FLAG: --image-service-endpoint="" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483202 2551 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483207 2551 flags.go:64] FLAG: --kube-api-burst="100" Jul 08 17:19:27.500217 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483210 2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483213 2551 flags.go:64] FLAG: --kube-api-qps="50" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483219 2551 flags.go:64] FLAG: --kube-reserved="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483222 2551 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483225 2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483230 2551 flags.go:64] FLAG: --kubelet-cgroups="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483233 2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483236 2551 flags.go:64] FLAG: --lock-file="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483239 2551 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483245 2551 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483249 2551 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483257 2551 flags.go:64] FLAG: --log-json-split-stream="false" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483260 2551 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483264 2551 flags.go:64] FLAG: --log-text-split-stream="false" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483267 2551 flags.go:64] FLAG: --logging-format="text" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483273 2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483278 2551 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483283 2551 flags.go:64] FLAG: --manifest-url="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483286 2551 flags.go:64] FLAG: --manifest-url-header="" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483291 2551 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483294 2551 flags.go:64] FLAG: --max-open-files="1000000" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483301 2551 flags.go:64] FLAG: --max-pods="110" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483305 2551 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483309 2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483314 2551 flags.go:64] FLAG: --memory-manager-policy="None" Jul 08 17:19:27.501384 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483317 2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483321 2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483324 2551 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483335 2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483361 2551 flags.go:64] FLAG: --node-status-max-images="50" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483368 2551 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483371 2551 flags.go:64] FLAG: --oom-score-adj="-999" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483374 2551 flags.go:64] FLAG: --pod-cidr="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483380 2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:ce93f6b4197b9c62a26a3131f8057ffe9c9379c5079a4f62e5919b75151394e5" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483388 2551 flags.go:64] FLAG: --pod-manifest-path="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483391 2551 flags.go:64] FLAG: --pod-max-pids="-1" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483394 2551 flags.go:64] FLAG: --pods-per-core="0" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483397 2551 flags.go:64] FLAG: --port="10250" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483401 2551 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483404 2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0fcb40b0a34f62216" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483408 2551 flags.go:64] FLAG: --qos-reserved="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483411 2551 flags.go:64] FLAG: --read-only-port="10255" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483414 2551 flags.go:64] FLAG: --register-node="true" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483420 2551 flags.go:64] FLAG: --register-with-taints="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483425 2551 flags.go:64] FLAG: --registry-burst="10" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483428 2551 flags.go:64] FLAG: --registry-qps="5" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483431 2551 flags.go:64] FLAG: --reserved-cpus="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483437 2551 flags.go:64] FLAG: --reserved-memory="" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483442 2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483447 2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 08 17:19:27.697568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483451 2551 flags.go:64] FLAG: --rotate-certificates="false" Jul 08 17:19:27.583266 ip-10-0-141-16 systemd[1]: Started Kubernetes Kubelet. Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483454 2551 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483458 2551 flags.go:64] FLAG: --runonce="false" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483462 2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483465 2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483473 2551 flags.go:64] FLAG: --seccomp-default="false" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483476 2551 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483480 2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483485 2551 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483488 2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483492 2551 flags.go:64] FLAG: --storage-driver-password="root" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483497 2551 flags.go:64] FLAG: --storage-driver-secure="false" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483500 2551 flags.go:64] FLAG: --storage-driver-table="stats" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483503 2551 flags.go:64] FLAG: --storage-driver-user="root" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483507 2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483513 2551 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483516 2551 flags.go:64] FLAG: --system-cgroups="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483519 2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483527 2551 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483530 2551 flags.go:64] FLAG: --tls-cert-file="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483533 2551 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483540 2551 flags.go:64] FLAG: --tls-min-version="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483543 2551 flags.go:64] FLAG: --tls-private-key-file="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483546 2551 flags.go:64] FLAG: --topology-manager-policy="none" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483563 2551 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 08 17:19:27.718688 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483569 2551 flags.go:64] FLAG: --topology-manager-scope="container" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483573 2551 flags.go:64] FLAG: --v="2" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483577 2551 flags.go:64] FLAG: --version="false" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483584 2551 flags.go:64] FLAG: --vmodule="" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483588 2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.483595 2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483750 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483754 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483758 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483763 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483766 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483769 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483776 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483779 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483782 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483787 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483790 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483793 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483796 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483801 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483805 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.719971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483808 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483813 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483816 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483819 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483824 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483827 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483830 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483833 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483838 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483841 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483844 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483846 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483851 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483855 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483858 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483862 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483865 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483868 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483871 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483873 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.720864 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483876 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483879 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483884 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483889 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483893 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483898 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483901 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483904 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483907 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483912 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483915 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483919 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483924 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483927 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483930 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483935 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483937 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483940 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483943 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.721582 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483949 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483952 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483955 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483958 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483963 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483967 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483971 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483977 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483981 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483985 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483990 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483992 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483995 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.483998 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484001 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484006 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484010 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484013 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484018 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.722249 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484021 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484024 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484029 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484032 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484035 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484038 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484041 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484046 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484049 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484051 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484056 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484059 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484061 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484068 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484071 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484074 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484077 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484082 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484085 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484087 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.723135 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484093 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484096 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484099 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484103 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484106 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.484109 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.484795 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.514479 2551 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.514501 2551 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.514533 2551 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.514541 2551 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514625 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514632 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514636 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514639 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.723964 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514643 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514646 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514649 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514652 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514655 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514658 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514661 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514664 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514666 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514669 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514672 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514675 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514678 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514682 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514685 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514688 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514691 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514695 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514698 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514701 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.724603 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514704 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514706 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514709 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514712 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514715 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514718 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514721 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514724 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514727 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514731 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514734 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514738 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514741 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514743 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514746 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514749 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514752 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514755 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514758 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514761 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.725312 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514764 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514767 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514769 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514772 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514776 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514778 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514781 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514784 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514788 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514791 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514794 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514798 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514802 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514807 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514810 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514813 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514816 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514819 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514821 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514824 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.726968 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514827 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514830 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514833 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514837 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514841 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514845 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514848 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514851 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514854 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514857 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514860 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514863 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514868 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514871 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514874 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514884 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514888 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514891 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.727689 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514894 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514896 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514899 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514902 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514905 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514908 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514911 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514914 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514917 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514920 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514922 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514925 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514928 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514931 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514933 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514937 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.729054 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.514939 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.514945 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515122 2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515129 2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515132 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515137 2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515141 2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515144 2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515148 2551 feature_gate.go:328] unrecognized feature gate: Example2 Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515150 2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515153 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515156 2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515159 2551 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515162 2551 feature_gate.go:328] unrecognized feature gate: Example Jul 08 17:19:27.729905 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515171 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515174 2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515177 2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515180 2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515182 2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515185 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515189 2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515192 2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515194 2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515197 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515200 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515203 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515205 2551 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515208 2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515212 2551 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515215 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515217 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515221 2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515224 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 08 17:19:27.914793 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515228 2551 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515232 2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515235 2551 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515238 2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515241 2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515244 2551 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515247 2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515250 2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515253 2551 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515256 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515259 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515263 2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515267 2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515270 2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515274 2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515278 2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515281 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515284 2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515287 2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515291 2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 08 17:19:27.915919 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515293 2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515296 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515299 2551 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515302 2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515305 2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515308 2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515311 2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515313 2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515316 2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515319 2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515322 2551 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515324 2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515327 2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515331 2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515334 2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515336 2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515339 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515342 2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515345 2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515347 2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 08 17:19:27.916740 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515350 2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515353 2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515356 2551 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515359 2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515362 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515365 2551 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515367 2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515371 2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515374 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515380 2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515382 2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515385 2551 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515388 2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515390 2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515393 2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515396 2551 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515398 2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515401 2551 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515404 2551 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515407 2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 08 17:19:27.917366 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515409 2551 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515412 2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515415 2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515418 2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515420 2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515423 2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515426 2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:27.515429 2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.515433 2551 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.515607 2551 server.go:962] "Client rotation is on, will bootstrap in background" Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.518148 2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.519707 2551 server.go:1019] "Starting client certificate rotation" Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.519826 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:19:27.918204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.519865 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.522992 2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.524312 2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.534637 2551 log.go:25] "Validated CRI v1 runtime API" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.536997 2551 log.go:25] "Validated CRI v1 image API" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.542085 2551 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.544662 2551 fs.go:135] Filesystem UUIDs: map[07667baa-8886-444b-94dc-3820f015451e:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 e4900509-f6c4-4658-9cc1-108f56f1a58e:/dev/nvme0n1p3] Jul 08 17:19:27.918685 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.544684 2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.551899 2551 manager.go:216] Machine: {Timestamp:2026-07-08 17:19:27.549471249 +0000 UTC m=+0.481282968 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102315 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec277abfaf453963b531bde245651f60 SystemUUID:ec277abf-af45-3963-b531-bde245651f60 BootID:337be2a9-6e51-4621-a9e5-8402ccd05d54 Filesystems:[{Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:40:86:2c:6f:cf Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:40:86:2c:6f:cf Speed:0 Mtu:9001} {Name:ovs-system MacAddress:8e:a8:5c:21:24:0f Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.552020 2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.552168 2551 manager.go:232] Version: {KernelVersion:5.14.0-570.125.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260630-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.553482 2551 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.553530 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.553544 2551 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-141-16.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.558762 2551 topology_manager.go:138] "Creating topology manager with none policy" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.558777 2551 container_manager_linux.go:309] "Creating device plugin manager" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.558795 2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.559878 2551 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.561425 2551 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565139 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565276 2551 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565317 2551 kubelet.go:486] "Attempting to sync node with API server" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565339 2551 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565363 2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 08 17:19:27.919211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565391 2551 kubelet.go:398] "Adding apiserver pod source" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565460 2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565980 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.565996 2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.570413 2551 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.575575 2551 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.575611 2551 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577827 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577862 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577871 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577877 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577890 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577898 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577910 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577915 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577926 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577935 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577949 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.577956 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.578631 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.578639 2551 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.580399 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-141-16.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.580396 2551 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.581533 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zmst8" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.582212 2551 server.go:1268] "Started kubelet" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.582289 2551 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.582376 2551 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.582468 2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.584087 2551 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.584865 2551 server.go:310] "Adding debug handlers to kubelet server" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.586978 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-zmst8" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.594855 2551 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.594964 2551 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-141-16.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.595510 2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.595558 2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.596011 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.594909 2551 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-141-16.ec2.internal.18c06025b992d814 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-141-16.ec2.internal,UID:ip-10-0-141-16.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-141-16.ec2.internal,},FirstTimestamp:2026-07-08 17:19:27.582181396 +0000 UTC m=+0.513993114,LastTimestamp:2026-07-08 17:19:27.582181396 +0000 UTC m=+0.513993114,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-141-16.ec2.internal,}" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596446 2551 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596447 2551 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596473 2551 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 08 17:19:27.920102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596684 2551 reconstruct.go:97] "Volume reconstruction finished" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596698 2551 reconciler.go:29] "Reconciler: start to sync state" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596877 2551 factory.go:55] Registering systemd factory Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.596892 2551 factory.go:223] Registration of the systemd container factory successfully Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597170 2551 factory.go:153] Registering CRI-O factory Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597187 2551 factory.go:223] Registration of the crio container factory successfully Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597259 2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597294 2551 factory.go:103] Registering Raw factory Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597308 2551 manager.go:1196] Started watching for new ooms in manager Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.597947 2551 manager.go:349] Starting recovery of all containers Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.615082 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.617255 2551 manager.go:354] Recovery completed Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.618585 2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-141-16.ec2.internal\" not found" node="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.621650 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.623780 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.623818 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.623829 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.624308 2551 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.624317 2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.624335 2551 state_mem.go:36] "Initialized new in-memory state store" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.626199 2551 policy_none.go:49] "None policy: Start" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.626218 2551 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.626229 2551 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.627328 2551 policy_none.go:47] "Start" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.628097 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.629075 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.629101 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.629114 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672347 2551 manager.go:337] "Starting Device Plugin manager" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.672393 2551 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672408 2551 server.go:87] "Starting device plugin registration server" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672763 2551 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672781 2551 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672846 2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672954 2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.672966 2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.673598 2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.673786 2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.713599 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.714921 2551 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.714967 2551 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.715004 2551 kubelet.go:2448] "Starting kubelet main sync loop" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.715052 2551 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.717614 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.775542 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.776608 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.776642 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.776653 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.776679 2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.783372 2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.921334 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.783393 2551 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-141-16.ec2.internal\": node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.800510 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.815666 2551 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal"] Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.815765 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.816815 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.816846 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.816857 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.820275 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.820413 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.820448 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821227 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821256 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821267 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821794 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821820 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.821834 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.826959 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.827407 2551 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.828490 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientMemory" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.828520 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasNoDiskPressure" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.828535 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeHasSufficientPID" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.844567 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-16.ec2.internal\" not found" node="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.850165 2551 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-141-16.ec2.internal\" not found" node="ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.898396 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c41a36ba272456960f43f5342563530e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-16.ec2.internal\" (UID: \"c41a36ba272456960f43f5342563530e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.898447 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.898468 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:27.923085 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:27.900561 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999575 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c41a36ba272456960f43f5342563530e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-16.ec2.internal\" (UID: \"c41a36ba272456960f43f5342563530e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999622 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999640 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999672 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999686 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/c41a36ba272456960f43f5342563530e-config\") pod \"kube-apiserver-proxy-ip-10-0-141-16.ec2.internal\" (UID: \"c41a36ba272456960f43f5342563530e\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:27.999710 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/513f8e21c222f58edf5aa4c1a8fccbb6-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal\" (UID: \"513f8e21c222f58edf5aa4c1a8fccbb6\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.000696 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.103247 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.101307 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.255220 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.150922 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.255220 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.153791 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.255220 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.201428 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.462466 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.301498 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.462466 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.397363 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:28.462466 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.402056 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:28.502517 2551 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-141-16.ec2.internal\" not found" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.518618 2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.518789 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.518832 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.518832 2551 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.569102 2551 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.596381 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.596385 2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.597551 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-07 17:14:27 +0000 UTC" deadline="2027-12-06 20:41:50.243931083 +0000 UTC" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.597592 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12387h22m21.646341499s" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.610858 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.611950 2551 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.617239 2551 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.621378 2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.634834 2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-r7kz8" Jul 08 17:19:28.651035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.643790 2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-r7kz8" Jul 08 17:19:28.825461 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:28.825429 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513f8e21c222f58edf5aa4c1a8fccbb6.slice/crio-233ef3fdc05860dee636043b4c4859cdd4879b1aa2d4e533037273e3ad19dbc8 WatchSource:0}: Error finding container 233ef3fdc05860dee636043b4c4859cdd4879b1aa2d4e533037273e3ad19dbc8: Status 404 returned error can't find the container with id 233ef3fdc05860dee636043b4c4859cdd4879b1aa2d4e533037273e3ad19dbc8 Jul 08 17:19:28.826187 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:28.826166 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc41a36ba272456960f43f5342563530e.slice/crio-3989a6c3d93906c309f049229dbe14bf29449aca06e9310d89865da9b94cb28d WatchSource:0}: Error finding container 3989a6c3d93906c309f049229dbe14bf29449aca06e9310d89865da9b94cb28d: Status 404 returned error can't find the container with id 3989a6c3d93906c309f049229dbe14bf29449aca06e9310d89865da9b94cb28d Jul 08 17:19:28.830573 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:28.830554 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:19:29.452941 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.452765 2551 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:29.566728 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.566695 2551 apiserver.go:52] "Watching apiserver" Jul 08 17:19:29.577088 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.576850 2551 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 08 17:19:29.577399 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.577372 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-wmtbj","openshift-network-operator/iptables-alerter-f6hgx","openshift-ovn-kubernetes/ovnkube-node-79xqw","kube-system/global-pull-secret-syncer-9lxj9","kube-system/konnectivity-agent-4fqdx","kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal","openshift-cluster-node-tuning-operator/tuned-hcdxn","openshift-dns/node-resolver-bbcrm","openshift-image-registry/node-ca-gw6q5","openshift-multus/multus-685fm","openshift-multus/multus-additional-cni-plugins-xzfvc","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal","openshift-multus/network-metrics-daemon-f9lcr"] Jul 08 17:19:29.580239 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.580212 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:29.580377 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.580329 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:29.582567 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.582526 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.586008 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.585115 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.586008 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.585701 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.586008 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.585800 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-5kv99\"" Jul 08 17:19:29.586008 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.585815 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 08 17:19:29.586008 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.585846 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.587707 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.587685 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.587803 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.587774 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:29.588431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.588065 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.588431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.588183 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.588431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.588183 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 08 17:19:29.588431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.588234 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 08 17:19:29.588431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.588318 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 08 17:19:29.589900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.589843 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-l68k6\"" Jul 08 17:19:29.590013 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.589911 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 08 17:19:29.590302 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.590282 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.594843 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.594612 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 08 17:19:29.594942 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.594917 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-b4qdq\"" Jul 08 17:19:29.595164 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.595146 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 08 17:19:29.595522 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.595502 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.598808 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.598253 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.598808 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.598626 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.599171 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.599153 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.599245 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.599176 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-dghgk\"" Jul 08 17:19:29.601281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.601035 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.601724 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.601542 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-bhbhr\"" Jul 08 17:19:29.601724 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.601611 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.602143 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.602125 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.604034 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.603814 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-d6gwx\"" Jul 08 17:19:29.604832 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.604223 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-685fm" Jul 08 17:19:29.604832 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.604653 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.605214 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.605198 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.605618 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.605603 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 08 17:19:29.606864 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.606558 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.607106 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.607083 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.607177 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.607165 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 08 17:19:29.607742 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.607723 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.607830 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.607742 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-p567b\"" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.608979 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k7lz4\" (UniqueName: \"kubernetes.io/projected/7a3a8b13-afe3-4a34-a186-1c6284e939bf-kube-api-access-k7lz4\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609017 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609044 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609071 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-log-socket\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609097 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-config\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609122 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3a8b13-afe3-4a34-a186-1c6284e939bf-tmp-dir\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609146 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-modprobe-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609170 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55j9b\" (UniqueName: \"kubernetes.io/projected/eeb714d9-9aa3-47d3-89ee-2855200dc928-kube-api-access-55j9b\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609197 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-etc-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609256 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-netd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609294 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-tmp\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609329 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/183fd0a6-e75d-459e-be3e-02ed31a476fb-agent-certs\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.609386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609361 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-netns\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609413 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609462 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-env-overrides\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609486 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609521 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-systemd\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609551 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-lib-modules\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609562 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609574 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-host\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609611 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-kubelet-config\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609641 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609665 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-bin\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609700 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysconfig\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609724 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-conf\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609747 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-run\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-dbus\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609794 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609825 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eeb714d9-9aa3-47d3-89ee-2855200dc928-iptables-alerter-script\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.610108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609913 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-systemd-units\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.609950 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-node-log\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610000 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-var-lib-kubelet\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610035 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-tuned\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610088 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/183fd0a6-e75d-459e-be3e-02ed31a476fb-konnectivity-ca\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610124 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eeb714d9-9aa3-47d3-89ee-2855200dc928-host-slash\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610168 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-slash\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610192 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-var-lib-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610216 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86803512-2772-4368-8135-2a7aed427df5-ovn-node-metrics-cert\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610258 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-script-lib\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610287 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3a8b13-afe3-4a34-a186-1c6284e939bf-hosts-file\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610334 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-kubelet\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610359 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610369 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-systemd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610411 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-ovn\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610453 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-kubernetes\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610479 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.610987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610538 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-sys\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.611840 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610570 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5mqxt\" (UniqueName: \"kubernetes.io/projected/86803512-2772-4368-8135-2a7aed427df5-kube-api-access-5mqxt\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.611840 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610624 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4d67\" (UniqueName: \"kubernetes.io/projected/98c759ce-709a-4b06-9e71-05cc5167c9a9-kube-api-access-x4d67\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.611840 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610784 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 08 17:19:29.611840 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.610800 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-jzw72\"" Jul 08 17:19:29.612623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.612602 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.612623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.612616 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 08 17:19:29.612769 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.612676 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:29.612769 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.612723 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-4c4dq\"" Jul 08 17:19:29.612992 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.612969 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 08 17:19:29.613089 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.613009 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 08 17:19:29.648814 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.648780 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:14:28 +0000 UTC" deadline="2027-12-08 22:36:42.823297504 +0000 UTC" Jul 08 17:19:29.648814 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.648813 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12437h17m13.174488378s" Jul 08 17:19:29.697664 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.697584 2551 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 08 17:19:29.711147 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711063 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-slash\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711147 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711107 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-var-lib-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711147 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711140 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86803512-2772-4368-8135-2a7aed427df5-ovn-node-metrics-cert\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711167 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3a8b13-afe3-4a34-a186-1c6284e939bf-hosts-file\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711213 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tt572\" (UniqueName: \"kubernetes.io/projected/8b987db0-50e6-488b-8045-090ef0800ad5-kube-api-access-tt572\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711286 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-var-lib-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711340 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-bin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711375 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-multus\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.711414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711408 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-kubelet\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711437 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-ovn\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711464 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-sys\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711489 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-os-release\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711519 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpf8v\" (UniqueName: \"kubernetes.io/projected/f8c404aa-32fa-47ec-8d9e-50ba17aed326-kube-api-access-zpf8v\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711563 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-cnibin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711589 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-kubelet\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711617 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4d67\" (UniqueName: \"kubernetes.io/projected/98c759ce-709a-4b06-9e71-05cc5167c9a9-kube-api-access-x4d67\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711640 2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 08 17:19:29.711755 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711722 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-slash\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711774 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-kubelet\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711822 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-ovn\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711646 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711879 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-sys\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711896 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.711931 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-log-socket\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712006 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7a3a8b13-afe3-4a34-a186-1c6284e939bf-hosts-file\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712053 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-log-socket\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712061 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-config\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712095 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-system-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712177 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-modprobe-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712263 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712257 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55j9b\" (UniqueName: \"kubernetes.io/projected/eeb714d9-9aa3-47d3-89ee-2855200dc928-kube-api-access-55j9b\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712289 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-system-cni-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712326 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cnibin\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712392 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712420 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-env-overrides\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712457 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-systemd\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712484 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-host\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712532 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-dbus\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712580 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712607 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/866f67bd-00c2-4ce4-93a9-36dc09df9314-host\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712632 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-bin\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712655 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-modprobe-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712667 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysconfig\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712692 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-run\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712717 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-k8s-cni-cncf-io\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712744 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.712904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712804 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-config\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712869 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712895 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-host\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712822 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712937 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-systemd-units\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712965 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-node-log\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.712991 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-var-lib-kubelet\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713019 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-tuned\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713045 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/183fd0a6-e75d-459e-be3e-02ed31a476fb-konnectivity-ca\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713076 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/866f67bd-00c2-4ce4-93a9-36dc09df9314-serviceca\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713103 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-72d8h\" (UniqueName: \"kubernetes.io/projected/866f67bd-00c2-4ce4-93a9-36dc09df9314-kube-api-access-72d8h\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713219 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713251 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eeb714d9-9aa3-47d3-89ee-2855200dc928-host-slash\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713277 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-script-lib\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713268 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-env-overrides\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-hostroot\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713350 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-systemd\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.713785 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713333 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-daemon-config\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713397 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-systemd-units\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713398 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-systemd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713435 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-node-log\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713435 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-kubernetes\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713470 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713483 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-kubernetes\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713500 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-etc-kubernetes\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713659 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-d\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713712 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-bin\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713741 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-dbus\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713756 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysconfig\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713808 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-run\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713817 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5mqxt\" (UniqueName: \"kubernetes.io/projected/86803512-2772-4368-8135-2a7aed427df5-kube-api-access-5mqxt\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713849 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/eeb714d9-9aa3-47d3-89ee-2855200dc928-host-slash\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713850 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k7lz4\" (UniqueName: \"kubernetes.io/projected/7a3a8b13-afe3-4a34-a186-1c6284e939bf-kube-api-access-k7lz4\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713858 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-var-lib-kubelet\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.713813 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-systemd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.714701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714077 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714113 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-conf-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714168 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-device-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714203 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714253 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3a8b13-afe3-4a34-a186-1c6284e939bf-tmp-dir\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714729 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-multus-certs\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.715518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.714791 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87629\" (UniqueName: \"kubernetes.io/projected/f5d4771c-285f-4d7a-a07b-559f0f784d42-kube-api-access-87629\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.718675 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.718641 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-etc-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719009 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/183fd0a6-e75d-459e-be3e-02ed31a476fb-konnectivity-ca\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719068 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-netd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719090 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-etc-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719104 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-tmp\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719141 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-run-openvswitch\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719143 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/183fd0a6-e75d-459e-be3e-02ed31a476fb-agent-certs\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719272 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-os-release\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719313 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-netns\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719349 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-lib-modules\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719412 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-cni-netd\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719417 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7a3a8b13-afe3-4a34-a186-1c6284e939bf-tmp-dir\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719482 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-kubelet-config\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719598 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-lib-modules\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719670 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719708 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-cni-binary-copy\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719724 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/38b897f5-aac6-4cb3-b2d2-605d3570f32e-kubelet-config\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.720910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719760 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719891 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7w28w\" (UniqueName: \"kubernetes.io/projected/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kube-api-access-7w28w\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719932 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720026 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-conf\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720059 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720091 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-socket-dir-parent\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720125 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-netns\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720209 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eeb714d9-9aa3-47d3-89ee-2855200dc928-iptables-alerter-script\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720241 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720277 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.719792 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-netns\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.720431 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720442 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-sysctl-conf\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.720514 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:19:30.220492793 +0000 UTC m=+3.152304518 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.720598 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/86803512-2772-4368-8135-2a7aed427df5-host-run-ovn-kubernetes\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.721990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.721144 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/eeb714d9-9aa3-47d3-89ee-2855200dc928-iptables-alerter-script\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.725576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.722342 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/86803512-2772-4368-8135-2a7aed427df5-ovnkube-script-lib\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.725637 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.725599 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-tmp\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.725698 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.725628 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/98c759ce-709a-4b06-9e71-05cc5167c9a9-etc-tuned\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.725761 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.725702 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/86803512-2772-4368-8135-2a7aed427df5-ovn-node-metrics-cert\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.727201 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.727148 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" event={"ID":"513f8e21c222f58edf5aa4c1a8fccbb6","Type":"ContainerStarted","Data":"233ef3fdc05860dee636043b4c4859cdd4879b1aa2d4e533037273e3ad19dbc8"} Jul 08 17:19:29.727478 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.727432 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/183fd0a6-e75d-459e-be3e-02ed31a476fb-agent-certs\") pod \"konnectivity-agent-4fqdx\" (UID: \"183fd0a6-e75d-459e-be3e-02ed31a476fb\") " pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.729876 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.729302 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" event={"ID":"c41a36ba272456960f43f5342563530e","Type":"ContainerStarted","Data":"3989a6c3d93906c309f049229dbe14bf29449aca06e9310d89865da9b94cb28d"} Jul 08 17:19:29.729989 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.729875 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55j9b\" (UniqueName: \"kubernetes.io/projected/eeb714d9-9aa3-47d3-89ee-2855200dc928-kube-api-access-55j9b\") pod \"iptables-alerter-f6hgx\" (UID: \"eeb714d9-9aa3-47d3-89ee-2855200dc928\") " pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.732647 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.732546 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4d67\" (UniqueName: \"kubernetes.io/projected/98c759ce-709a-4b06-9e71-05cc5167c9a9-kube-api-access-x4d67\") pod \"tuned-hcdxn\" (UID: \"98c759ce-709a-4b06-9e71-05cc5167c9a9\") " pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.736149 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.736103 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5mqxt\" (UniqueName: \"kubernetes.io/projected/86803512-2772-4368-8135-2a7aed427df5-kube-api-access-5mqxt\") pod \"ovnkube-node-79xqw\" (UID: \"86803512-2772-4368-8135-2a7aed427df5\") " pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.736946 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.736921 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-k7lz4\" (UniqueName: \"kubernetes.io/projected/7a3a8b13-afe3-4a34-a186-1c6284e939bf-kube-api-access-k7lz4\") pod \"node-resolver-bbcrm\" (UID: \"7a3a8b13-afe3-4a34-a186-1c6284e939bf\") " pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.737283 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.737258 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:29.737325 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.737294 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:29.737325 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.737307 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:29.737393 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.737380 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:19:30.237361838 +0000 UTC m=+3.169173556 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:29.821164 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821124 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-os-release\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821164 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821169 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821188 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-cni-binary-copy\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821205 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821229 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7w28w\" (UniqueName: \"kubernetes.io/projected/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kube-api-access-7w28w\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821271 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-socket-dir-parent\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821281 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-os-release\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821294 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-netns\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821324 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821349 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821353 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-etc-selinux\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821394 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-netns\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.821425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821356 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-socket-dir-parent\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821412 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-registration-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821438 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tt572\" (UniqueName: \"kubernetes.io/projected/8b987db0-50e6-488b-8045-090ef0800ad5-kube-api-access-tt572\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821469 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-bin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821495 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-multus\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821542 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-os-release\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821563 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-bin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821569 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpf8v\" (UniqueName: \"kubernetes.io/projected/f8c404aa-32fa-47ec-8d9e-50ba17aed326-kube-api-access-zpf8v\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821620 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-cnibin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821648 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-kubelet\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821666 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-os-release\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821636 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-cni-multus\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821677 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821694 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-cnibin\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821750 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-var-lib-kubelet\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821757 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821802 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-system-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821829 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-system-cni-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821854 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cnibin\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821855 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-tuning-conf-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821890 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-socket-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821902 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cnibin\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821898 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821921 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-cni-binary-copy\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821929 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821940 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f8c404aa-32fa-47ec-8d9e-50ba17aed326-system-cni-dir\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821953 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.821974 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.821980 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/866f67bd-00c2-4ce4-93a9-36dc09df9314-host\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822008 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-k8s-cni-cncf-io\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:29.822033 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:19:30.322012057 +0000 UTC m=+3.253823763 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822050 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-k8s-cni-cncf-io\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822058 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822067 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/866f67bd-00c2-4ce4-93a9-36dc09df9314-host\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.822877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822087 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822093 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822087 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-system-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822127 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/866f67bd-00c2-4ce4-93a9-36dc09df9314-serviceca\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822139 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822145 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-sys-fs\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822179 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-72d8h\" (UniqueName: \"kubernetes.io/projected/866f67bd-00c2-4ce4-93a9-36dc09df9314-kube-api-access-72d8h\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822206 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822254 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-hostroot\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822282 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-daemon-config\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822290 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-cni-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822299 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-hostroot\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822311 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-etc-kubernetes\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822366 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-etc-kubernetes\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822425 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-conf-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822450 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-device-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822470 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-multus-certs\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822477 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/f8c404aa-32fa-47ec-8d9e-50ba17aed326-cni-binary-copy\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.823674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822512 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-87629\" (UniqueName: \"kubernetes.io/projected/f5d4771c-285f-4d7a-a07b-559f0f784d42-kube-api-access-87629\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.824426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822535 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/866f67bd-00c2-4ce4-93a9-36dc09df9314-serviceca\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.824426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822543 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-host-run-multus-certs\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.824426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822543 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/27bae590-c0d8-4e6c-8209-63e1c3b4493e-device-dir\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.824426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822513 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-conf-dir\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.824426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.822813 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/f5d4771c-285f-4d7a-a07b-559f0f784d42-multus-daemon-config\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.833582 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.833545 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7w28w\" (UniqueName: \"kubernetes.io/projected/27bae590-c0d8-4e6c-8209-63e1c3b4493e-kube-api-access-7w28w\") pod \"aws-ebs-csi-driver-node-fxp29\" (UID: \"27bae590-c0d8-4e6c-8209-63e1c3b4493e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.834503 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.834474 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tt572\" (UniqueName: \"kubernetes.io/projected/8b987db0-50e6-488b-8045-090ef0800ad5-kube-api-access-tt572\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:29.834639 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.834620 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87629\" (UniqueName: \"kubernetes.io/projected/f5d4771c-285f-4d7a-a07b-559f0f784d42-kube-api-access-87629\") pod \"multus-685fm\" (UID: \"f5d4771c-285f-4d7a-a07b-559f0f784d42\") " pod="openshift-multus/multus-685fm" Jul 08 17:19:29.835269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.835249 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-72d8h\" (UniqueName: \"kubernetes.io/projected/866f67bd-00c2-4ce4-93a9-36dc09df9314-kube-api-access-72d8h\") pod \"node-ca-gw6q5\" (UID: \"866f67bd-00c2-4ce4-93a9-36dc09df9314\") " pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.835647 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.835630 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpf8v\" (UniqueName: \"kubernetes.io/projected/f8c404aa-32fa-47ec-8d9e-50ba17aed326-kube-api-access-zpf8v\") pod \"multus-additional-cni-plugins-xzfvc\" (UID: \"f8c404aa-32fa-47ec-8d9e-50ba17aed326\") " pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.898839 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.898802 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-f6hgx" Jul 08 17:19:29.917238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.917200 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:29.925686 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.925660 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bbcrm" Jul 08 17:19:29.933377 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.933353 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:29.940140 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.940119 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" Jul 08 17:19:29.950019 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.949999 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-gw6q5" Jul 08 17:19:29.957771 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.957751 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-685fm" Jul 08 17:19:29.964678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.964633 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" Jul 08 17:19:29.968552 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.968526 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" Jul 08 17:19:29.998525 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:29.998486 2551 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:30.127167 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.127132 2551 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 08 17:19:30.226492 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.226373 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:30.226655 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.226574 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:30.226655 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.226649 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:19:31.226631552 +0000 UTC m=+4.158443272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:30.326971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.326932 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:30.327151 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.327093 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:30.327227 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327149 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:30.327227 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327178 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:30.327227 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327193 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:30.327227 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327210 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:30.327430 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327268 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:19:31.3272489 +0000 UTC m=+4.259060618 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:30.327430 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.327294 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:19:31.327282284 +0000 UTC m=+4.259094002 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:30.650065 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.649967 2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-07 17:14:28 +0000 UTC" deadline="2028-03-23 05:42:14.50487569 +0000 UTC" Jul 08 17:19:30.650065 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.650012 2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14964h22m43.854866677s" Jul 08 17:19:30.716013 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.715978 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:30.716635 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.715978 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:30.716635 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.716114 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:30.716635 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:30.716121 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:30.716635 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.716204 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:30.716635 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:30.716302 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:30.975979 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.975947 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeeb714d9_9aa3_47d3_89ee_2855200dc928.slice/crio-fb9ceb4879dc36a3721b88387193d15a8779fdff5d7fa8a31ef36e758a102fa7 WatchSource:0}: Error finding container fb9ceb4879dc36a3721b88387193d15a8779fdff5d7fa8a31ef36e758a102fa7: Status 404 returned error can't find the container with id fb9ceb4879dc36a3721b88387193d15a8779fdff5d7fa8a31ef36e758a102fa7 Jul 08 17:19:30.976945 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.976918 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8c404aa_32fa_47ec_8d9e_50ba17aed326.slice/crio-0ecb6f214554eccc914d10f744c8d4395e1b382e5bf84adb9d632b0774440b22 WatchSource:0}: Error finding container 0ecb6f214554eccc914d10f744c8d4395e1b382e5bf84adb9d632b0774440b22: Status 404 returned error can't find the container with id 0ecb6f214554eccc914d10f744c8d4395e1b382e5bf84adb9d632b0774440b22 Jul 08 17:19:30.981144 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.981018 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod183fd0a6_e75d_459e_be3e_02ed31a476fb.slice/crio-2d092ee31b00cfe26a3fb0f955ac26f9fc4117ae3294a4c886d81f5ecebe3140 WatchSource:0}: Error finding container 2d092ee31b00cfe26a3fb0f955ac26f9fc4117ae3294a4c886d81f5ecebe3140: Status 404 returned error can't find the container with id 2d092ee31b00cfe26a3fb0f955ac26f9fc4117ae3294a4c886d81f5ecebe3140 Jul 08 17:19:30.982555 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.982436 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod98c759ce_709a_4b06_9e71_05cc5167c9a9.slice/crio-da5a2cc2d794df1425231fe0741ac79850d363d1f174053244981792c65f90bb WatchSource:0}: Error finding container da5a2cc2d794df1425231fe0741ac79850d363d1f174053244981792c65f90bb: Status 404 returned error can't find the container with id da5a2cc2d794df1425231fe0741ac79850d363d1f174053244981792c65f90bb Jul 08 17:19:30.983142 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.983120 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod86803512_2772_4368_8135_2a7aed427df5.slice/crio-51490de6f7660f95550b67aed18308d3218cbcbcd64a2c83902b422038b89fc9 WatchSource:0}: Error finding container 51490de6f7660f95550b67aed18308d3218cbcbcd64a2c83902b422038b89fc9: Status 404 returned error can't find the container with id 51490de6f7660f95550b67aed18308d3218cbcbcd64a2c83902b422038b89fc9 Jul 08 17:19:30.984227 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.984184 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5d4771c_285f_4d7a_a07b_559f0f784d42.slice/crio-4a431e889acff8416ccfe2202db1785a910ec5008a72e64eb6944445ca94e1f3 WatchSource:0}: Error finding container 4a431e889acff8416ccfe2202db1785a910ec5008a72e64eb6944445ca94e1f3: Status 404 returned error can't find the container with id 4a431e889acff8416ccfe2202db1785a910ec5008a72e64eb6944445ca94e1f3 Jul 08 17:19:30.985635 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:19:30.985608 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod866f67bd_00c2_4ce4_93a9_36dc09df9314.slice/crio-9e9ea04047b62ab2eea20f8eb5d75d0d54de0c7272d55e9ac5a3bcfab5373283 WatchSource:0}: Error finding container 9e9ea04047b62ab2eea20f8eb5d75d0d54de0c7272d55e9ac5a3bcfab5373283: Status 404 returned error can't find the container with id 9e9ea04047b62ab2eea20f8eb5d75d0d54de0c7272d55e9ac5a3bcfab5373283 Jul 08 17:19:31.235380 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.235287 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:31.235544 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.235470 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:31.235643 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.235579 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:19:33.235555334 +0000 UTC m=+6.167367058 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:31.336885 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.336692 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:31.337021 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.336913 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:31.337021 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.336843 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:31.337124 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.337035 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:31.337124 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.337047 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:31.337124 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.337063 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:31.337124 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.337064 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:19:33.337043275 +0000 UTC m=+6.268854978 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:31.337124 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:31.337103 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:19:33.337091027 +0000 UTC m=+6.268902749 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:31.744770 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.744672 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" event={"ID":"27bae590-c0d8-4e6c-8209-63e1c3b4493e","Type":"ContainerStarted","Data":"190ba80b19ed0073f2247e67d6f5cba324d9c6b01b1416abe71b67d8b0b970cd"} Jul 08 17:19:31.751275 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.751130 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bbcrm" event={"ID":"7a3a8b13-afe3-4a34-a186-1c6284e939bf","Type":"ContainerStarted","Data":"2802f0afcafaccb35e33efe02cfd253bb27299f97d332f888806818886b78e56"} Jul 08 17:19:31.763597 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.763459 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gw6q5" event={"ID":"866f67bd-00c2-4ce4-93a9-36dc09df9314","Type":"ContainerStarted","Data":"9e9ea04047b62ab2eea20f8eb5d75d0d54de0c7272d55e9ac5a3bcfab5373283"} Jul 08 17:19:31.771418 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.771338 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6hgx" event={"ID":"eeb714d9-9aa3-47d3-89ee-2855200dc928","Type":"ContainerStarted","Data":"fb9ceb4879dc36a3721b88387193d15a8779fdff5d7fa8a31ef36e758a102fa7"} Jul 08 17:19:31.782680 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.781672 2551 generic.go:356] "Generic (PLEG): container finished" podID="513f8e21c222f58edf5aa4c1a8fccbb6" containerID="07f6040bb30b883cd029b4765243f39f8f027e41d30bcdd4ea3f26a881e301b5" exitCode=0 Jul 08 17:19:31.782680 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.781765 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" event={"ID":"513f8e21c222f58edf5aa4c1a8fccbb6","Type":"ContainerDied","Data":"07f6040bb30b883cd029b4765243f39f8f027e41d30bcdd4ea3f26a881e301b5"} Jul 08 17:19:31.791754 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.791635 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" event={"ID":"c41a36ba272456960f43f5342563530e","Type":"ContainerStarted","Data":"51aa4bc0a7624d0f6931ddcf61e510f76b01edfb952a8e68a9ade71a3c7e3de0"} Jul 08 17:19:31.800126 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.799353 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-685fm" event={"ID":"f5d4771c-285f-4d7a-a07b-559f0f784d42","Type":"ContainerStarted","Data":"4a431e889acff8416ccfe2202db1785a910ec5008a72e64eb6944445ca94e1f3"} Jul 08 17:19:31.804791 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.804760 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"51490de6f7660f95550b67aed18308d3218cbcbcd64a2c83902b422038b89fc9"} Jul 08 17:19:31.810456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.810426 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" event={"ID":"98c759ce-709a-4b06-9e71-05cc5167c9a9","Type":"ContainerStarted","Data":"da5a2cc2d794df1425231fe0741ac79850d363d1f174053244981792c65f90bb"} Jul 08 17:19:31.817706 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.816512 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-141-16.ec2.internal" podStartSLOduration=3.816492224 podStartE2EDuration="3.816492224s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:31.816359425 +0000 UTC m=+4.748171151" watchObservedRunningTime="2026-07-08 17:19:31.816492224 +0000 UTC m=+4.748303945" Jul 08 17:19:31.818225 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.818130 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4fqdx" event={"ID":"183fd0a6-e75d-459e-be3e-02ed31a476fb","Type":"ContainerStarted","Data":"2d092ee31b00cfe26a3fb0f955ac26f9fc4117ae3294a4c886d81f5ecebe3140"} Jul 08 17:19:31.822898 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:31.822871 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerStarted","Data":"0ecb6f214554eccc914d10f744c8d4395e1b382e5bf84adb9d632b0774440b22"} Jul 08 17:19:32.716364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:32.716328 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:32.716600 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:32.716468 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:32.716928 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:32.716909 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:32.717018 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:32.717000 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:32.717088 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:32.717029 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:32.717137 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:32.717107 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:32.832434 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:32.832394 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" event={"ID":"513f8e21c222f58edf5aa4c1a8fccbb6","Type":"ContainerStarted","Data":"b5919de05d26459d7732305c3985e01addc2e243cf73dffd8ded4464b8928623"} Jul 08 17:19:33.255073 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:33.254941 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:33.255269 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.255152 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:33.255269 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.255246 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:19:37.255225315 +0000 UTC m=+10.187037028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:33.355514 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:33.355460 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:33.355705 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:33.355554 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:33.355768 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.355723 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:33.355768 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.355745 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:33.355768 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.355758 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:33.355919 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.355815 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:19:37.355796746 +0000 UTC m=+10.287608470 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:33.356244 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.356225 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:33.356313 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:33.356282 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:19:37.356265727 +0000 UTC m=+10.288077429 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:34.715506 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:34.715278 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:34.715506 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:34.715321 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:34.715506 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:34.715426 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:34.716072 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:34.715531 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:34.716072 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:34.715601 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:34.716072 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:34.715677 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:36.716252 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:36.716221 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:36.716252 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:36.716239 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:36.716788 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:36.716358 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:36.716788 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:36.716359 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:36.716788 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:36.716439 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:36.716788 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:36.716535 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:37.288410 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:37.288372 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:37.288613 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.288579 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:37.288694 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.288647 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:19:45.288627111 +0000 UTC m=+18.220438829 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:37.389628 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:37.389569 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:37.389813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:37.389662 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:37.389813 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389759 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:37.389913 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389832 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:19:45.389812213 +0000 UTC m=+18.321623937 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:37.389913 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389885 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:37.389913 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389912 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:37.390057 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389926 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:37.390057 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:37.389980 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:19:45.389965505 +0000 UTC m=+18.321777208 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:38.716236 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:38.715612 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:38.716236 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:38.715752 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:38.716236 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:38.716176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:38.716236 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:38.716201 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:38.716820 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:38.716284 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:38.716820 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:38.716391 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:40.715960 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:40.715927 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:40.715960 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:40.715959 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:40.716442 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:40.716047 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:40.716442 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:40.716064 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:40.716442 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:40.716168 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:40.716442 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:40.716267 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:42.715566 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:42.715525 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:42.715566 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:42.715553 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:42.716057 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:42.715547 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:42.716057 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:42.715672 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:42.716057 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:42.715784 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:42.716057 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:42.715869 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:44.715444 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:44.715406 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:44.715966 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:44.715406 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:44.715966 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:44.715567 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:44.715966 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:44.715429 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:44.715966 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:44.715603 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:44.715966 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:44.715671 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:45.356403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:45.356359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:45.356601 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.356500 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:45.356601 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.356552 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:20:01.356539568 +0000 UTC m=+34.288351269 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:19:45.457147 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:45.457111 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:45.457291 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:45.457275 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:45.457366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457290 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:19:45.457366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457312 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:19:45.457366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457323 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:45.457519 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457376 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:20:01.457362974 +0000 UTC m=+34.389174674 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:19:45.457519 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457404 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:45.457519 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:45.457459 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:01.457441443 +0000 UTC m=+34.389253160 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:19:46.715407 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:46.715357 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:46.715407 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:46.715390 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:46.715407 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:46.715407 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:46.716027 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:46.715528 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:46.716027 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:46.715657 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:46.716027 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:46.715758 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:48.715258 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:48.715220 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:48.715752 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:48.715221 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:48.715752 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:48.715220 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:48.715752 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:48.715494 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:48.715752 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:48.715352 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:48.715752 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:48.715557 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:50.716155 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.715766 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:50.716994 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.715823 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:50.716994 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:50.716216 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:50.716994 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:50.716290 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:50.716994 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.715823 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:50.716994 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:50.716408 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:50.874857 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.874820 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" event={"ID":"27bae590-c0d8-4e6c-8209-63e1c3b4493e","Type":"ContainerStarted","Data":"638a28eee3b61bb9c75d0a0351b25250a124a212de700031039a223093f77cbe"} Jul 08 17:19:50.876040 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.876016 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bbcrm" event={"ID":"7a3a8b13-afe3-4a34-a186-1c6284e939bf","Type":"ContainerStarted","Data":"cd6d6d90fc2d555d0543b5af2a10c9c2855a1fd3f4bfb124fd7a7f4a15967b8f"} Jul 08 17:19:50.877321 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.877296 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-gw6q5" event={"ID":"866f67bd-00c2-4ce4-93a9-36dc09df9314","Type":"ContainerStarted","Data":"466f20d092e8825cca1e0d42e5014a49307048cb44eb64c27721dd1620e7b7f0"} Jul 08 17:19:50.878578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.878558 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-685fm" event={"ID":"f5d4771c-285f-4d7a-a07b-559f0f784d42","Type":"ContainerStarted","Data":"83adad15f307bf53a60b4a7ccf57527e49c7995e40ac08f893d0f1d05f26a864"} Jul 08 17:19:50.881329 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881308 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"b64ddc5726c94f214df2109cb33ae905a9fa74a76676c9fe3ae79752180da1c7"} Jul 08 17:19:50.881412 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881369 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"e90b7d2fe4384cdd45ee3e7a46abe9d8cb12ac19974cd6fff5aac4ffc33c3a20"} Jul 08 17:19:50.881448 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881411 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"29f701191e4213b128e44781e598ccbabdb56c3023cce5f44c1e43517db451de"} Jul 08 17:19:50.881502 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881453 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"bdb4d83f6263eb76db30b0efdfc2daeb39f72de35d400f319049cf546960f699"} Jul 08 17:19:50.881502 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881496 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"45a09b08851def8af326e27255017a1d97b1276e98d5ab6d82861ffe0273eace"} Jul 08 17:19:50.881584 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.881535 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"69dc5363b59f861abb7eae73249e2a8f337c50dea0d45a7e12aa7ca43c386138"} Jul 08 17:19:50.882673 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.882655 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" event={"ID":"98c759ce-709a-4b06-9e71-05cc5167c9a9","Type":"ContainerStarted","Data":"474d0a83b9b33eef24a941e03f778d8e4adca4aa6c7aa9ff8e8291a389be28be"} Jul 08 17:19:50.883990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.883958 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4fqdx" event={"ID":"183fd0a6-e75d-459e-be3e-02ed31a476fb","Type":"ContainerStarted","Data":"b018c6b1fae62181d69765401c2bb22bef03e75a9d59367e0b59febdbdf4960d"} Jul 08 17:19:50.885259 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.885233 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="7cd4c43d52b919e5183c611d70bbc37e1b891059db37b352a4a960f6c2d5f67c" exitCode=0 Jul 08 17:19:50.885339 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.885267 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"7cd4c43d52b919e5183c611d70bbc37e1b891059db37b352a4a960f6c2d5f67c"} Jul 08 17:19:50.892371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.892335 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-bbcrm" podStartSLOduration=4.077590068 podStartE2EDuration="22.892324836s" totalImagesPullingTime="18.814734768s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:50.892323439 +0000 UTC m=+23.824135161" watchObservedRunningTime="2026-07-08 17:19:50.892324836 +0000 UTC m=+23.824136558" Jul 08 17:19:50.892646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.892624 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-141-16.ec2.internal" podStartSLOduration=22.892618524 podStartE2EDuration="22.892618524s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:32.850073576 +0000 UTC m=+5.781885303" watchObservedRunningTime="2026-07-08 17:19:50.892618524 +0000 UTC m=+23.824430249" Jul 08 17:19:50.907395 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.907338 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-gw6q5" podStartSLOduration=4.0925400100000004 podStartE2EDuration="22.907319839s" totalImagesPullingTime="18.814779829s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:50.906718831 +0000 UTC m=+23.838530555" watchObservedRunningTime="2026-07-08 17:19:50.907319839 +0000 UTC m=+23.839131565" Jul 08 17:19:50.926241 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.926190 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-685fm" podStartSLOduration=4.030206989 podStartE2EDuration="22.926174512s" totalImagesPullingTime="18.895967523s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:50.926143595 +0000 UTC m=+23.857955319" watchObservedRunningTime="2026-07-08 17:19:50.926174512 +0000 UTC m=+23.857986236" Jul 08 17:19:50.954467 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.954406 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4fqdx" podStartSLOduration=12.261717698 podStartE2EDuration="22.954389983s" totalImagesPullingTime="10.692672285s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:50.954349396 +0000 UTC m=+23.886161118" watchObservedRunningTime="2026-07-08 17:19:50.954389983 +0000 UTC m=+23.886201706" Jul 08 17:19:50.971777 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:50.971686 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-hcdxn" podStartSLOduration=4.127021655 podStartE2EDuration="22.971666014s" totalImagesPullingTime="18.844644359s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:50.971235922 +0000 UTC m=+23.903047646" watchObservedRunningTime="2026-07-08 17:19:50.971666014 +0000 UTC m=+23.903477737" Jul 08 17:19:51.496056 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.496031 2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 08 17:19:51.685886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.685724 2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-08T17:19:51.496050798Z","UUID":"6dd302de-88c6-4510-90cd-66e959b3ee68","Handler":null,"Name":"","Endpoint":""} Jul 08 17:19:51.687376 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.687353 2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 08 17:19:51.687488 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.687392 2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 08 17:19:51.890335 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.890303 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" event={"ID":"27bae590-c0d8-4e6c-8209-63e1c3b4493e","Type":"ContainerStarted","Data":"7c32fe4f7eb111b39acacdb55b9b5fbfbc2c348921af23c79fdd04e722414631"} Jul 08 17:19:51.891968 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:51.891942 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-f6hgx" event={"ID":"eeb714d9-9aa3-47d3-89ee-2855200dc928","Type":"ContainerStarted","Data":"50e53135524cc1ed5671b597cb05ca821723efc00db0b368386d5786105c309f"} Jul 08 17:19:52.716023 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:52.715991 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:52.716145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:52.716008 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:52.716145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:52.716036 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:52.716145 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:52.716129 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:52.716286 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:52.716229 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:52.716369 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:52.716347 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:52.896466 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:52.896430 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"d51b51dbf6bc68095e063965dcc256d3faa2f60c448aa7f5e70aae6166cf95a9"} Jul 08 17:19:53.900634 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:53.900372 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" event={"ID":"27bae590-c0d8-4e6c-8209-63e1c3b4493e","Type":"ContainerStarted","Data":"c5aadfc938077c073a0a50abbc2d74e118845cf611e0f3074eb2d27a659761b5"} Jul 08 17:19:53.921139 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:53.921073 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-fxp29" podStartSLOduration=4.425735863 podStartE2EDuration="25.921052414s" totalImagesPullingTime="21.495316551s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:53.920456376 +0000 UTC m=+26.852268123" watchObservedRunningTime="2026-07-08 17:19:53.921052414 +0000 UTC m=+26.852864138" Jul 08 17:19:53.921393 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:53.921363 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-f6hgx" podStartSLOduration=7.070947427 podStartE2EDuration="25.921354485s" totalImagesPullingTime="18.850407058s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:51.91015704 +0000 UTC m=+24.841968765" watchObservedRunningTime="2026-07-08 17:19:53.921354485 +0000 UTC m=+26.853166209" Jul 08 17:19:54.715925 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:54.715882 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:54.716102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:54.715897 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:54.716102 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:54.716014 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:54.716102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:54.716035 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:54.716254 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:54.716152 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:54.716254 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:54.716232 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:54.921987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:54.921950 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:54.922678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:54.922661 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:55.904955 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:55.904929 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:55.905425 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:55.905409 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4fqdx" Jul 08 17:19:56.715964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.715725 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:56.715964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.715782 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:56.716644 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:56.715967 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:56.716644 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.715782 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:56.716644 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:56.716039 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:56.716644 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:56.716115 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:56.910554 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.910515 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" event={"ID":"86803512-2772-4368-8135-2a7aed427df5","Type":"ContainerStarted","Data":"2605d48e69c8b831bddda81d800f570eb8e512284dfa9d4841874d511c87599e"} Jul 08 17:19:56.910843 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.910821 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:56.912371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.912346 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="3675ecb63267227dd393b445bd9f90066c8c6b56555dc6d09a54e77e27e98d15" exitCode=0 Jul 08 17:19:56.912490 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.912426 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"3675ecb63267227dd393b445bd9f90066c8c6b56555dc6d09a54e77e27e98d15"} Jul 08 17:19:56.926570 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.926549 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:56.940762 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:56.940710 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" podStartSLOduration=10.016928073 podStartE2EDuration="28.940696679s" totalImagesPullingTime="18.923768606s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:19:56.940591778 +0000 UTC m=+29.872403501" watchObservedRunningTime="2026-07-08 17:19:56.940696679 +0000 UTC m=+29.872508405" Jul 08 17:19:57.825314 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.825289 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bbcrm_7a3a8b13-afe3-4a34-a186-1c6284e939bf/dns-node-resolver/0.log" Jul 08 17:19:57.915470 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.915445 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:57.915615 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.915575 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:57.930402 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.930380 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:19:57.983017 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.982988 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f9lcr"] Jul 08 17:19:57.983141 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.983108 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:57.983217 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:57.983200 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:57.986290 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.986268 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wmtbj"] Jul 08 17:19:57.986393 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.986355 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:57.986439 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:57.986423 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:19:57.999574 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.999544 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lxj9"] Jul 08 17:19:57.999684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:57.999652 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:57.999778 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:57.999750 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:58.598215 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:58.598186 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gw6q5_866f67bd-00c2-4ce4-93a9-36dc09df9314/node-ca/0.log" Jul 08 17:19:58.919720 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:58.919689 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="5e45b9d08596ef277307b4757eceef8195c0a4e41108d357d3c22ac29c42cdca" exitCode=0 Jul 08 17:19:58.920088 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:58.919763 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"5e45b9d08596ef277307b4757eceef8195c0a4e41108d357d3c22ac29c42cdca"} Jul 08 17:19:59.716251 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:59.716075 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:19:59.716390 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:59.716075 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:19:59.716390 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:59.716358 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:19:59.716491 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:19:59.716141 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:19:59.716491 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:59.716417 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:19:59.716586 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:19:59.716531 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:00.926246 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:00.926216 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="538f267a64e03f8e5a295367aaf26c7d03111caed1020149f3f70024d8389906" exitCode=0 Jul 08 17:20:00.926741 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:00.926274 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"538f267a64e03f8e5a295367aaf26c7d03111caed1020149f3f70024d8389906"} Jul 08 17:20:01.372489 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.372390 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:01.372636 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.372566 2551 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 08 17:20:01.372636 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.372628 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret podName:38b897f5-aac6-4cb3-b2d2-605d3570f32e nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.372613349 +0000 UTC m=+66.304425053 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret") pod "global-pull-secret-syncer-9lxj9" (UID: "38b897f5-aac6-4cb3-b2d2-605d3570f32e") : object "kube-system"/"original-pull-secret" not registered Jul 08 17:20:01.473329 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.473294 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:01.473522 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.473359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:01.473522 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473448 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 08 17:20:01.473522 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473470 2551 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 08 17:20:01.473522 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473497 2551 projected.go:196] Error preparing data for projected volume kube-api-access-gq8rb for pod openshift-network-diagnostics/network-check-target-wmtbj: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:20:01.473724 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473448 2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:20:01.473724 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473542 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb podName:b71226d0-4ca5-4a86-acf8-95b991a31aec nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.47352795 +0000 UTC m=+66.405339654 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-gq8rb" (UniqueName: "kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb") pod "network-check-target-wmtbj" (UID: "b71226d0-4ca5-4a86-acf8-95b991a31aec") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 08 17:20:01.473724 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.473582 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs podName:8b987db0-50e6-488b-8045-090ef0800ad5 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:33.473568264 +0000 UTC m=+66.405379964 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs") pod "network-metrics-daemon-f9lcr" (UID: "8b987db0-50e6-488b-8045-090ef0800ad5") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 08 17:20:01.715553 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.715451 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:01.715553 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.715462 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:01.715743 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:01.715600 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:01.715743 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.715605 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:01.715743 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.715686 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:01.715833 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:01.715759 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:03.715836 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:03.715807 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:03.715836 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:03.715824 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:03.716469 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:03.715965 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:03.716469 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:03.716044 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:03.716469 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:03.716063 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:03.716469 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:03.716150 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:05.716220 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:05.715976 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:05.716220 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:05.715976 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:05.716220 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:05.716113 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:05.716220 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:05.716142 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:05.716871 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:05.716274 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:05.716871 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:05.716369 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:07.716705 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:07.716673 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:07.717170 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:07.716772 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:07.717170 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:07.716821 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:07.717170 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:07.716944 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:07.717170 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:07.716979 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:07.717170 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:07.717022 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:09.715912 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:09.715876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:09.716366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:09.715876 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:09.716366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:09.715985 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:09.716366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:09.716078 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:09.716366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:09.715889 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:09.716366 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:09.716154 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:09.952897 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:09.952866 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="51cb926f1b0e081a5a3d4af2d0fdf248db9990038367220fa65be2a58d0b8421" exitCode=0 Jul 08 17:20:09.952897 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:09.952909 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"51cb926f1b0e081a5a3d4af2d0fdf248db9990038367220fa65be2a58d0b8421"} Jul 08 17:20:10.958133 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:10.958100 2551 generic.go:356] "Generic (PLEG): container finished" podID="f8c404aa-32fa-47ec-8d9e-50ba17aed326" containerID="3dc1a2a1c5d0104bf4465f0e289a80e9fd66755a59f5ed222c23633a33cc5ee7" exitCode=0 Jul 08 17:20:10.958537 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:10.958147 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerDied","Data":"3dc1a2a1c5d0104bf4465f0e289a80e9fd66755a59f5ed222c23633a33cc5ee7"} Jul 08 17:20:11.715580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:11.715538 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:11.715580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:11.715570 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:11.715833 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:11.715577 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:11.715833 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:11.715647 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:11.715833 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:11.715707 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:11.715833 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:11.715771 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:11.964125 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:11.964084 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" event={"ID":"f8c404aa-32fa-47ec-8d9e-50ba17aed326","Type":"ContainerStarted","Data":"c1b7a67c23c4b57e498819e8bf89c2c6662db21bda260429eb6a863b73936347"} Jul 08 17:20:11.990203 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:11.990100 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-xzfvc" podStartSLOduration=9.474842625 podStartE2EDuration="43.990082609s" totalImagesPullingTime="34.515239984s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:11.989630991 +0000 UTC m=+44.921442715" watchObservedRunningTime="2026-07-08 17:20:11.990082609 +0000 UTC m=+44.921894332" Jul 08 17:20:13.716218 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:13.716186 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:13.716678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:13.716186 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:13.716678 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:13.716302 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:13.716678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:13.716194 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:13.716678 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:13.716383 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:13.716678 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:13.716494 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:15.715405 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:15.715366 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:15.715823 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:15.715366 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:15.715823 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:15.715503 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:15.715823 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:15.715495 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:15.715823 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:15.715576 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:15.715823 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:15.715633 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:17.716964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:17.716923 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:17.717446 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:17.717025 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:17.717446 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:17.717111 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:17.717446 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:17.717230 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:17.717446 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:17.717236 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:17.717446 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:17.717313 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:19.715422 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:19.715389 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:19.715919 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:19.715390 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:19.715919 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:19.715522 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-f9lcr" podUID="8b987db0-50e6-488b-8045-090ef0800ad5" Jul 08 17:20:19.715919 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:19.715402 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:19.715919 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:19.715586 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wmtbj" podUID="b71226d0-4ca5-4a86-acf8-95b991a31aec" Jul 08 17:20:19.715919 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:19.715659 2551 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-9lxj9" podUID="38b897f5-aac6-4cb3-b2d2-605d3570f32e" Jul 08 17:20:20.877734 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.877708 2551 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-141-16.ec2.internal" event="NodeReady" Jul 08 17:20:20.878107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.877828 2551 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 08 17:20:20.929364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.929334 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fk242"] Jul 08 17:20:20.947260 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.947235 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-z2hmv"] Jul 08 17:20:20.947377 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.947356 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:20.950451 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.950413 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-bqdcg\"" Jul 08 17:20:20.950451 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.950451 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 08 17:20:20.950668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.950423 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 08 17:20:20.950668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.950511 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 08 17:20:20.950668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.950611 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 08 17:20:20.973281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.973253 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fk242"] Jul 08 17:20:20.973423 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.973292 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2hmv"] Jul 08 17:20:20.973423 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.973366 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:20.978825 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.978803 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 08 17:20:20.979313 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.979285 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 08 17:20:20.979440 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.979343 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-sbxhn\"" Jul 08 17:20:20.979440 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:20.979343 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 08 17:20:21.013033 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.012998 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.013196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013059 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26lcn\" (UniqueName: \"kubernetes.io/projected/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-api-access-26lcn\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.013196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013097 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26cb2c9d-dc86-4938-916d-8151673d9e04-cert\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.013196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013125 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gl8q\" (UniqueName: \"kubernetes.io/projected/26cb2c9d-dc86-4938-916d-8151673d9e04-kube-api-access-4gl8q\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.013196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013170 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-data-volume\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.013196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013190 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-crio-socket\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.013352 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.013208 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.027437 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.027407 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4rpfv"] Jul 08 17:20:21.055491 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.055447 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rpfv"] Jul 08 17:20:21.055646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.055579 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.058530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.058503 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 08 17:20:21.058530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.058523 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-rc7v6\"" Jul 08 17:20:21.058722 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.058523 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 08 17:20:21.113700 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113667 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.113700 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113705 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-982zs\" (UniqueName: \"kubernetes.io/projected/b475e0f8-649b-48ac-ba1d-c1810d8f635b-kube-api-access-982zs\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113739 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26lcn\" (UniqueName: \"kubernetes.io/projected/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-api-access-26lcn\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113761 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26cb2c9d-dc86-4938-916d-8151673d9e04-cert\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113843 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gl8q\" (UniqueName: \"kubernetes.io/projected/26cb2c9d-dc86-4938-916d-8151673d9e04-kube-api-access-4gl8q\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113880 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b475e0f8-649b-48ac-ba1d-c1810d8f635b-tmp-dir\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113907 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b475e0f8-649b-48ac-ba1d-c1810d8f635b-config-volume\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.113972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.113937 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-data-volume\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.114226 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114010 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b475e0f8-649b-48ac-ba1d-c1810d8f635b-metrics-tls\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.114226 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114054 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-crio-socket\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.114226 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114082 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.114226 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114223 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-data-volume\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.114423 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114237 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-crio-socket\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.114631 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.114614 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.118023 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.117995 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.118023 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.118013 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/26cb2c9d-dc86-4938-916d-8151673d9e04-cert\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.122954 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.122920 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gl8q\" (UniqueName: \"kubernetes.io/projected/26cb2c9d-dc86-4938-916d-8151673d9e04-kube-api-access-4gl8q\") pod \"ingress-canary-z2hmv\" (UID: \"26cb2c9d-dc86-4938-916d-8151673d9e04\") " pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.123075 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.122920 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-26lcn\" (UniqueName: \"kubernetes.io/projected/5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85-kube-api-access-26lcn\") pod \"insights-runtime-extractor-fk242\" (UID: \"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85\") " pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.215203 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215118 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-982zs\" (UniqueName: \"kubernetes.io/projected/b475e0f8-649b-48ac-ba1d-c1810d8f635b-kube-api-access-982zs\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.215203 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215190 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b475e0f8-649b-48ac-ba1d-c1810d8f635b-tmp-dir\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.215377 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215217 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b475e0f8-649b-48ac-ba1d-c1810d8f635b-config-volume\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.215377 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215239 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b475e0f8-649b-48ac-ba1d-c1810d8f635b-metrics-tls\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.215619 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215597 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/b475e0f8-649b-48ac-ba1d-c1810d8f635b-tmp-dir\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.215832 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.215812 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/b475e0f8-649b-48ac-ba1d-c1810d8f635b-config-volume\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.217453 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.217436 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/b475e0f8-649b-48ac-ba1d-c1810d8f635b-metrics-tls\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.223944 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.223922 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-982zs\" (UniqueName: \"kubernetes.io/projected/b475e0f8-649b-48ac-ba1d-c1810d8f635b-kube-api-access-982zs\") pod \"dns-default-4rpfv\" (UID: \"b475e0f8-649b-48ac-ba1d-c1810d8f635b\") " pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.257951 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.257915 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fk242" Jul 08 17:20:21.282059 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.282033 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-z2hmv" Jul 08 17:20:21.364316 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.364271 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:21.487963 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.487886 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-z2hmv"] Jul 08 17:20:21.490633 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.490601 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fk242"] Jul 08 17:20:21.492494 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:21.492453 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod26cb2c9d_dc86_4938_916d_8151673d9e04.slice/crio-ad234509b8fe415720477fef1b2e0d960d95eca6bd1cabeb37abd734abcf3448 WatchSource:0}: Error finding container ad234509b8fe415720477fef1b2e0d960d95eca6bd1cabeb37abd734abcf3448: Status 404 returned error can't find the container with id ad234509b8fe415720477fef1b2e0d960d95eca6bd1cabeb37abd734abcf3448 Jul 08 17:20:21.493710 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:21.493693 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bc76c93_40ad_42c0_a5b2_06c9ab5f5e85.slice/crio-77eabeaddfab5fbec4678a1d795a08ad4563709f88041aad7567784940db31f8 WatchSource:0}: Error finding container 77eabeaddfab5fbec4678a1d795a08ad4563709f88041aad7567784940db31f8: Status 404 returned error can't find the container with id 77eabeaddfab5fbec4678a1d795a08ad4563709f88041aad7567784940db31f8 Jul 08 17:20:21.520519 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.520381 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4rpfv"] Jul 08 17:20:21.522764 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:21.522740 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb475e0f8_649b_48ac_ba1d_c1810d8f635b.slice/crio-bbe01d86136278808704659037457dde3524941ec2cb91b53fecb64b4bc971ec WatchSource:0}: Error finding container bbe01d86136278808704659037457dde3524941ec2cb91b53fecb64b4bc971ec: Status 404 returned error can't find the container with id bbe01d86136278808704659037457dde3524941ec2cb91b53fecb64b4bc971ec Jul 08 17:20:21.719016 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.718979 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:21.719016 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.719022 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:21.719248 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.719022 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:21.722610 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722589 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:20:21.722610 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722618 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:20:21.722801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722598 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:20:21.722801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722591 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:20:21.722801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722698 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j7ksx\"" Jul 08 17:20:21.722949 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.722843 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8rq8b\"" Jul 08 17:20:21.987043 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.986956 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rpfv" event={"ID":"b475e0f8-649b-48ac-ba1d-c1810d8f635b","Type":"ContainerStarted","Data":"bbe01d86136278808704659037457dde3524941ec2cb91b53fecb64b4bc971ec"} Jul 08 17:20:21.988085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.988055 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2hmv" event={"ID":"26cb2c9d-dc86-4938-916d-8151673d9e04","Type":"ContainerStarted","Data":"ad234509b8fe415720477fef1b2e0d960d95eca6bd1cabeb37abd734abcf3448"} Jul 08 17:20:21.989174 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:21.989152 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk242" event={"ID":"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85","Type":"ContainerStarted","Data":"77eabeaddfab5fbec4678a1d795a08ad4563709f88041aad7567784940db31f8"} Jul 08 17:20:25.279173 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.279144 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r"] Jul 08 17:20:25.281235 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.281215 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:25.283959 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.283940 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 08 17:20:25.284081 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.284044 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-nbr9x\"" Jul 08 17:20:25.290712 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.290678 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r"] Jul 08 17:20:25.346051 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.346009 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91e4425a-6440-404f-8ab7-7dfd90760ead-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-54p7r\" (UID: \"91e4425a-6440-404f-8ab7-7dfd90760ead\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:25.447079 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.447033 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91e4425a-6440-404f-8ab7-7dfd90760ead-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-54p7r\" (UID: \"91e4425a-6440-404f-8ab7-7dfd90760ead\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:25.449796 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.449766 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/91e4425a-6440-404f-8ab7-7dfd90760ead-tls-certificates\") pod \"prometheus-operator-admission-webhook-d559d66d5-54p7r\" (UID: \"91e4425a-6440-404f-8ab7-7dfd90760ead\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:25.591235 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:25.591190 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:26.898113 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:26.898084 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r"] Jul 08 17:20:27.006822 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.006741 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" event={"ID":"91e4425a-6440-404f-8ab7-7dfd90760ead","Type":"ContainerStarted","Data":"996d72d4946b398e12e12b5b13bedb4a00dada919b3d0517825b89a2191e0721"} Jul 08 17:20:27.008197 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.008170 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-z2hmv" event={"ID":"26cb2c9d-dc86-4938-916d-8151673d9e04","Type":"ContainerStarted","Data":"ed59f9f3f7e092143fa9f4e85c752d06a2a291abc3262fa44cf6ec598f748cdc"} Jul 08 17:20:27.009800 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.009773 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk242" event={"ID":"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85","Type":"ContainerStarted","Data":"205efd77cd4128521eecdb845ac0a758cbe88cbb2366d32cea77469a26bf565c"} Jul 08 17:20:27.011392 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.011370 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rpfv" event={"ID":"b475e0f8-649b-48ac-ba1d-c1810d8f635b","Type":"ContainerStarted","Data":"2ff526d1ee3d51d11d38c2f17864536697fd152f1d12285e0089bb9b4387bdcb"} Jul 08 17:20:27.011516 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.011417 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4rpfv" event={"ID":"b475e0f8-649b-48ac-ba1d-c1810d8f635b","Type":"ContainerStarted","Data":"112d1eef233303e0b3d580f8fe260468bc7ba082fd0f7eace3ab43301f3b1c81"} Jul 08 17:20:27.011591 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.011529 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:27.025917 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.025865 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-z2hmv" podStartSLOduration=1.781462157 podStartE2EDuration="7.025847895s" totalImagesPullingTime="5.244385738s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:27.025703254 +0000 UTC m=+59.957514977" watchObservedRunningTime="2026-07-08 17:20:27.025847895 +0000 UTC m=+59.957659620" Jul 08 17:20:27.046601 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:27.046397 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4rpfv" podStartSLOduration=0.831870196 podStartE2EDuration="6.046376061s" totalImagesPullingTime="5.214505865s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:21 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:27.045901225 +0000 UTC m=+59.977712950" watchObservedRunningTime="2026-07-08 17:20:27.046376061 +0000 UTC m=+59.978187783" Jul 08 17:20:28.016955 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:28.016870 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk242" event={"ID":"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85","Type":"ContainerStarted","Data":"a69412422731b153441e16fa6a9859b15654812b92bfdbf4eac925a8248fcbad"} Jul 08 17:20:29.020817 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.020785 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" event={"ID":"91e4425a-6440-404f-8ab7-7dfd90760ead","Type":"ContainerStarted","Data":"ec61d7dfc79596320db3bf092bafa5e57eb74dca7e9f04bfd65f11790e5f8085"} Jul 08 17:20:29.021204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.020988 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:29.025816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.025794 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" Jul 08 17:20:29.037344 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.037306 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-d559d66d5-54p7r" podStartSLOduration=2.874836428 podStartE2EDuration="4.037294626s" totalImagesPullingTime="1.162458198s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:29.036745602 +0000 UTC m=+61.968557325" watchObservedRunningTime="2026-07-08 17:20:29.037294626 +0000 UTC m=+61.969106348" Jul 08 17:20:29.273114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.273033 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:20:29.305752 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.305721 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:20:29.305913 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.305870 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.308978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.308942 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 17:20:29.309120 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.308985 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-528mx\"" Jul 08 17:20:29.310359 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310256 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 17:20:29.310359 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310268 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 17:20:29.310359 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310278 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 17:20:29.310615 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310588 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 17:20:29.310615 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310608 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 17:20:29.310710 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.310620 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 17:20:29.354238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.354205 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4"] Jul 08 17:20:29.375735 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.375703 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4"] Jul 08 17:20:29.375971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.375854 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.378930 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.378899 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 08 17:20:29.378930 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.378908 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 08 17:20:29.378930 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.378921 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 08 17:20:29.379184 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.379003 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 08 17:20:29.379286 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.379259 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-bkw7v\"" Jul 08 17:20:29.379388 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.379303 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 08 17:20:29.387550 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387526 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.387651 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387563 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdpfr\" (UniqueName: \"kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.387651 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387641 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.387749 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387702 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.387814 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387789 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.387905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.387829 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.488726 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488683 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.488902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488746 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.488902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488780 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d03d347-1977-4785-968c-02670b56b10e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.488902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488825 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.488902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488871 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488944 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.488974 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdpfr\" (UniqueName: \"kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489015 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.489114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489044 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dtlmn\" (UniqueName: \"kubernetes.io/projected/2d03d347-1977-4785-968c-02670b56b10e-kube-api-access-dtlmn\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.489114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489093 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489657 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489630 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489937 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.489969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.489956 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.491777 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.491756 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.492122 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.492087 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.497671 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.497636 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdpfr\" (UniqueName: \"kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr\") pod \"console-5c99cb68c4-mdb5k\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.590291 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.590255 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.590291 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.590295 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dtlmn\" (UniqueName: \"kubernetes.io/projected/2d03d347-1977-4785-968c-02670b56b10e-kube-api-access-dtlmn\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.590573 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.590384 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.590573 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.590414 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d03d347-1977-4785-968c-02670b56b10e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.591256 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.591228 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2d03d347-1977-4785-968c-02670b56b10e-metrics-client-ca\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.593039 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.593012 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-tls\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.593153 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.593037 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2d03d347-1977-4785-968c-02670b56b10e-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.599903 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.599881 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dtlmn\" (UniqueName: \"kubernetes.io/projected/2d03d347-1977-4785-968c-02670b56b10e-kube-api-access-dtlmn\") pod \"prometheus-operator-567f7b8b7c-28mm4\" (UID: \"2d03d347-1977-4785-968c-02670b56b10e\") " pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.617076 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.617045 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:29.686427 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.686387 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" Jul 08 17:20:29.937975 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:29.937885 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-79xqw" Jul 08 17:20:30.280270 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:30.280225 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4"] Jul 08 17:20:30.282798 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:30.282763 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d03d347_1977_4785_968c_02670b56b10e.slice/crio-ec79a5ae5397a5ff17b4ba6b84c3fc7d9532f128cb52e5853a361a631dc68e1a WatchSource:0}: Error finding container ec79a5ae5397a5ff17b4ba6b84c3fc7d9532f128cb52e5853a361a631dc68e1a: Status 404 returned error can't find the container with id ec79a5ae5397a5ff17b4ba6b84c3fc7d9532f128cb52e5853a361a631dc68e1a Jul 08 17:20:30.298357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:30.298335 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:20:30.301896 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:30.301869 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7820f4da_bfbc_43c5_bbfe_fb53c6678c72.slice/crio-1a2bbb525d5b42c53bb72207652a2700e3ffc1880f2ac1c9017840925e59fd5c WatchSource:0}: Error finding container 1a2bbb525d5b42c53bb72207652a2700e3ffc1880f2ac1c9017840925e59fd5c: Status 404 returned error can't find the container with id 1a2bbb525d5b42c53bb72207652a2700e3ffc1880f2ac1c9017840925e59fd5c Jul 08 17:20:31.030435 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:31.030398 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" event={"ID":"2d03d347-1977-4785-968c-02670b56b10e","Type":"ContainerStarted","Data":"ec79a5ae5397a5ff17b4ba6b84c3fc7d9532f128cb52e5853a361a631dc68e1a"} Jul 08 17:20:31.033356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:31.033294 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fk242" event={"ID":"5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85","Type":"ContainerStarted","Data":"0d82e9c7fd72dca309aa2c03681b575aa3da1a45bd63ad95bab4b1170056f393"} Jul 08 17:20:31.035358 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:31.035323 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c99cb68c4-mdb5k" event={"ID":"7820f4da-bfbc-43c5-bbfe-fb53c6678c72","Type":"ContainerStarted","Data":"1a2bbb525d5b42c53bb72207652a2700e3ffc1880f2ac1c9017840925e59fd5c"} Jul 08 17:20:31.054899 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:31.054247 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fk242" podStartSLOduration=2.510796728 podStartE2EDuration="11.054228405s" totalImagesPullingTime="8.543431677s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:20:20 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:31.053363154 +0000 UTC m=+63.985174877" watchObservedRunningTime="2026-07-08 17:20:31.054228405 +0000 UTC m=+63.986040129" Jul 08 17:20:32.041662 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:32.041636 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" event={"ID":"2d03d347-1977-4785-968c-02670b56b10e","Type":"ContainerStarted","Data":"7a35d0dd7ff1fb2dca51065239ef164f9bff8c0dddb7d49d282eec65a63ead00"} Jul 08 17:20:33.047291 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.047249 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" event={"ID":"2d03d347-1977-4785-968c-02670b56b10e","Type":"ContainerStarted","Data":"8b807341b9d669e5f5b6a01c76738a5ef7e36d0aa1b3bef0b534368d63bd02cc"} Jul 08 17:20:33.079590 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.079519 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-567f7b8b7c-28mm4" podStartSLOduration=2.459745265 podStartE2EDuration="4.079501716s" totalImagesPullingTime="1.619756451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:33.079212343 +0000 UTC m=+66.011024066" watchObservedRunningTime="2026-07-08 17:20:33.079501716 +0000 UTC m=+66.011313417" Jul 08 17:20:33.422206 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.422163 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:33.425686 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.425660 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 08 17:20:33.435211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.435178 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/38b897f5-aac6-4cb3-b2d2-605d3570f32e-original-pull-secret\") pod \"global-pull-secret-syncer-9lxj9\" (UID: \"38b897f5-aac6-4cb3-b2d2-605d3570f32e\") " pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:33.439014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.438980 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-9lxj9" Jul 08 17:20:33.522617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.522579 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:33.522786 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.522640 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:33.525612 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.525585 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 08 17:20:33.525743 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.525624 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 08 17:20:33.535412 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.535388 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8b987db0-50e6-488b-8045-090ef0800ad5-metrics-certs\") pod \"network-metrics-daemon-f9lcr\" (UID: \"8b987db0-50e6-488b-8045-090ef0800ad5\") " pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:33.536110 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.536088 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 08 17:20:33.546162 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.546135 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gq8rb\" (UniqueName: \"kubernetes.io/projected/b71226d0-4ca5-4a86-acf8-95b991a31aec-kube-api-access-gq8rb\") pod \"network-check-target-wmtbj\" (UID: \"b71226d0-4ca5-4a86-acf8-95b991a31aec\") " pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:33.732065 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.732030 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-j7ksx\"" Jul 08 17:20:33.736783 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.736760 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-8rq8b\"" Jul 08 17:20:33.740133 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.740114 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-f9lcr" Jul 08 17:20:33.744635 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.744612 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:33.864148 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.864113 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-9lxj9"] Jul 08 17:20:33.870238 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:33.870202 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38b897f5_aac6_4cb3_b2d2_605d3570f32e.slice/crio-11ce91962c435bb5c7450427190b266d0468a69b3bb9ee747ace0959644e3c0e WatchSource:0}: Error finding container 11ce91962c435bb5c7450427190b266d0468a69b3bb9ee747ace0959644e3c0e: Status 404 returned error can't find the container with id 11ce91962c435bb5c7450427190b266d0468a69b3bb9ee747ace0959644e3c0e Jul 08 17:20:33.923515 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.923446 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wmtbj"] Jul 08 17:20:33.926343 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:33.926317 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb71226d0_4ca5_4a86_acf8_95b991a31aec.slice/crio-3519c388654457585fad393dd018a62a34c8b21e324525b1f73652f9e0571027 WatchSource:0}: Error finding container 3519c388654457585fad393dd018a62a34c8b21e324525b1f73652f9e0571027: Status 404 returned error can't find the container with id 3519c388654457585fad393dd018a62a34c8b21e324525b1f73652f9e0571027 Jul 08 17:20:33.936705 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:33.936682 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-f9lcr"] Jul 08 17:20:33.939614 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:33.939587 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8b987db0_50e6_488b_8045_090ef0800ad5.slice/crio-8f7d67f6abdb20b358748ef91d2e48579f28835341703fdb26be8fe823caf99f WatchSource:0}: Error finding container 8f7d67f6abdb20b358748ef91d2e48579f28835341703fdb26be8fe823caf99f: Status 404 returned error can't find the container with id 8f7d67f6abdb20b358748ef91d2e48579f28835341703fdb26be8fe823caf99f Jul 08 17:20:34.051887 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.051806 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wmtbj" event={"ID":"b71226d0-4ca5-4a86-acf8-95b991a31aec","Type":"ContainerStarted","Data":"3519c388654457585fad393dd018a62a34c8b21e324525b1f73652f9e0571027"} Jul 08 17:20:34.053299 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.053270 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c99cb68c4-mdb5k" event={"ID":"7820f4da-bfbc-43c5-bbfe-fb53c6678c72","Type":"ContainerStarted","Data":"a195c35c30dce1a4e85bd29450893469987f63e58c6800d04769ddf9faa7e5fd"} Jul 08 17:20:34.054363 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.054341 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9lcr" event={"ID":"8b987db0-50e6-488b-8045-090ef0800ad5","Type":"ContainerStarted","Data":"8f7d67f6abdb20b358748ef91d2e48579f28835341703fdb26be8fe823caf99f"} Jul 08 17:20:34.055340 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.055322 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lxj9" event={"ID":"38b897f5-aac6-4cb3-b2d2-605d3570f32e","Type":"ContainerStarted","Data":"11ce91962c435bb5c7450427190b266d0468a69b3bb9ee747ace0959644e3c0e"} Jul 08 17:20:34.072799 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.072752 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5c99cb68c4-mdb5k" podStartSLOduration=1.609627203 podStartE2EDuration="5.072738603s" totalImagesPullingTime="3.4631114s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:34.071739853 +0000 UTC m=+67.003551577" watchObservedRunningTime="2026-07-08 17:20:34.072738603 +0000 UTC m=+67.004550368" Jul 08 17:20:34.738039 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.737046 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s"] Jul 08 17:20:34.740715 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.740630 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.746381 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.745966 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-7m7mx\"" Jul 08 17:20:34.746381 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.746093 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 08 17:20:34.746381 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.746206 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:20:34.746655 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.746395 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s"] Jul 08 17:20:34.750050 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.750029 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f"] Jul 08 17:20:34.756069 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.756050 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.761216 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.761033 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\"" Jul 08 17:20:34.761793 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.761771 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-hgqbh\"" Jul 08 17:20:34.762813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.762650 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\"" Jul 08 17:20:34.763990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.763849 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\"" Jul 08 17:20:34.767616 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.766836 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f"] Jul 08 17:20:34.779978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.779013 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-jdjtz"] Jul 08 17:20:34.782549 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.782526 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.786318 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.786083 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 08 17:20:34.786318 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.786097 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 08 17:20:34.787678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.787504 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-cvxn8\"" Jul 08 17:20:34.787678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.787640 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 08 17:20:34.832269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832173 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40e73073-ca8f-4948-b620-ac5e25b50c28-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.832269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832224 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832297 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130c551-3d33-4a3a-982f-58795364bbad-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832332 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832376 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832412 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45g6l\" (UniqueName: \"kubernetes.io/projected/2130c551-3d33-4a3a-982f-58795364bbad-kube-api-access-45g6l\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832455 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ptpz2\" (UniqueName: \"kubernetes.io/projected/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-api-access-ptpz2\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.832556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832538 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.832868 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832579 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.832868 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.832629 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934716 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934773 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-sys\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934803 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934828 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934857 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934895 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40e73073-ca8f-4948-b620-ac5e25b50c28-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934932 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.934964 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvr4s\" (UniqueName: \"kubernetes.io/projected/90cd18a2-d4f5-440c-86b5-85922498bf45-kube-api-access-tvr4s\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935005 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130c551-3d33-4a3a-982f-58795364bbad-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935037 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935087 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-root\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935144 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935113 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935163 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935208 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45g6l\" (UniqueName: \"kubernetes.io/projected/2130c551-3d33-4a3a-982f-58795364bbad-kube-api-access-45g6l\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935232 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-metrics-client-ca\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935262 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-wtmp\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935304 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-textfile\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935332 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ptpz2\" (UniqueName: \"kubernetes.io/projected/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-api-access-ptpz2\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.935859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.935370 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.936182 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:34.935871 2551 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found Jul 08 17:20:34.936182 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:34.935932 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls podName:40e73073-ca8f-4948-b620-ac5e25b50c28 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:35.435912013 +0000 UTC m=+68.367723713 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls") pod "kube-state-metrics-7c9dbb8b7d-t4n7f" (UID: "40e73073-ca8f-4948-b620-ac5e25b50c28") : secret "kube-state-metrics-tls" not found Jul 08 17:20:34.936182 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:34.936152 2551 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 08 17:20:34.936336 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:34.936204 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls podName:2130c551-3d33-4a3a-982f-58795364bbad nodeName:}" failed. No retries permitted until 2026-07-08 17:20:35.436188481 +0000 UTC m=+68.368000207 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls") pod "openshift-state-metrics-54cf96dfcf-qq27s" (UID: "2130c551-3d33-4a3a-982f-58795364bbad") : secret "openshift-state-metrics-tls" not found Jul 08 17:20:34.936336 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.936208 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.936447 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.936411 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/40e73073-ca8f-4948-b620-ac5e25b50c28-volume-directive-shadow\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.936619 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.936527 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/40e73073-ca8f-4948-b620-ac5e25b50c28-metrics-client-ca\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.936975 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.936933 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/2130c551-3d33-4a3a-982f-58795364bbad-metrics-client-ca\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.938712 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.938689 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.940231 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.940189 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:34.949993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.949953 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ptpz2\" (UniqueName: \"kubernetes.io/projected/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-api-access-ptpz2\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:34.955316 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:34.955290 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-45g6l\" (UniqueName: \"kubernetes.io/projected/2130c551-3d33-4a3a-982f-58795364bbad-kube-api-access-45g6l\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036558 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvr4s\" (UniqueName: \"kubernetes.io/projected/90cd18a2-d4f5-440c-86b5-85922498bf45-kube-api-access-tvr4s\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036672 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-root\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036703 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036771 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-metrics-client-ca\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036820 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-wtmp\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036850 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-textfile\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036890 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-sys\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036928 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037296 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.036962 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037678 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-metrics-client-ca\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037684 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-accelerators-collector-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037710 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-wtmp\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037760 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-sys\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:35.037795 2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:35.037873 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls podName:90cd18a2-d4f5-440c-86b5-85922498bf45 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:35.537846156 +0000 UTC m=+68.469657870 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls") pod "node-exporter-jdjtz" (UID: "90cd18a2-d4f5-440c-86b5-85922498bf45") : secret "node-exporter-tls" not found Jul 08 17:20:35.037964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037866 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-textfile\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.038321 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.037657 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/90cd18a2-d4f5-440c-86b5-85922498bf45-root\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.041190 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.041161 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.049938 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.049905 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvr4s\" (UniqueName: \"kubernetes.io/projected/90cd18a2-d4f5-440c-86b5-85922498bf45-kube-api-access-tvr4s\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.440269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.440235 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:35.440269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.440304 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:35.443433 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.443382 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/40e73073-ca8f-4948-b620-ac5e25b50c28-kube-state-metrics-tls\") pod \"kube-state-metrics-7c9dbb8b7d-t4n7f\" (UID: \"40e73073-ca8f-4948-b620-ac5e25b50c28\") " pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:35.444026 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.443984 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/2130c551-3d33-4a3a-982f-58795364bbad-openshift-state-metrics-tls\") pod \"openshift-state-metrics-54cf96dfcf-qq27s\" (UID: \"2130c551-3d33-4a3a-982f-58795364bbad\") " pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:35.541017 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.540926 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:35.541277 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:35.541112 2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 08 17:20:35.541277 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:35.541182 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls podName:90cd18a2-d4f5-440c-86b5-85922498bf45 nodeName:}" failed. No retries permitted until 2026-07-08 17:20:36.541161176 +0000 UTC m=+69.472972880 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls") pod "node-exporter-jdjtz" (UID: "90cd18a2-d4f5-440c-86b5-85922498bf45") : secret "node-exporter-tls" not found Jul 08 17:20:35.654213 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.654176 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" Jul 08 17:20:35.670446 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.670212 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" Jul 08 17:20:35.825583 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.825534 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s"] Jul 08 17:20:35.830340 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:35.830290 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2130c551_3d33_4a3a_982f_58795364bbad.slice/crio-d37ea812d370c9113d1cece7526209e7179803239ec167f73190457e08eb8b8c WatchSource:0}: Error finding container d37ea812d370c9113d1cece7526209e7179803239ec167f73190457e08eb8b8c: Status 404 returned error can't find the container with id d37ea812d370c9113d1cece7526209e7179803239ec167f73190457e08eb8b8c Jul 08 17:20:35.837995 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.837967 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:20:35.841803 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.841782 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.845692 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.845888 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.846091 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.846272 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.846462 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.847102 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.847317 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.847381 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.847317 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 17:20:35.847969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.847605 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5bkkz\"" Jul 08 17:20:35.871756 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.870227 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f"] Jul 08 17:20:35.871756 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.871599 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:20:35.873522 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:35.873469 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40e73073_ca8f_4948_b620_ac5e25b50c28.slice/crio-b6f80f1f63c76848fe3e012793e3abdf0920b5f1aa12b48e2f0078bd37466554 WatchSource:0}: Error finding container b6f80f1f63c76848fe3e012793e3abdf0920b5f1aa12b48e2f0078bd37466554: Status 404 returned error can't find the container with id b6f80f1f63c76848fe3e012793e3abdf0920b5f1aa12b48e2f0078bd37466554 Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948435 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948513 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948541 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948566 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rcnnh\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948619 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948679 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948715 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948741 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948809 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948839 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948869 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948900 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:35.949064 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:35.948925 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.049936 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.049865 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.049936 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.049938 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.049976 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050003 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050042 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050070 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050102 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050151 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050173 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050206 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050234 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050258 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050282 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rcnnh\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050775 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.050953 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.050927 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.051670 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.051637 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.054085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.054060 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.054813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.054588 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.054813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.054699 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.054813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.054742 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.055055 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.054984 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.055307 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.055282 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.055589 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.055555 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.055788 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.055758 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.056212 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.056191 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.059364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.059339 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rcnnh\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh\") pod \"alertmanager-main-0\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.071495 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.071444 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9lcr" event={"ID":"8b987db0-50e6-488b-8045-090ef0800ad5","Type":"ContainerStarted","Data":"97d59d2addc8bab2b9f8a6b13ab845c11499c39e68bf16019995bfdcd0c1a606"} Jul 08 17:20:36.071631 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.071525 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-f9lcr" event={"ID":"8b987db0-50e6-488b-8045-090ef0800ad5","Type":"ContainerStarted","Data":"ae49d86c4a4b416ed0a657370bd0fb6984accc7df5b80ceaafb2c0bc418e782e"} Jul 08 17:20:36.073046 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.073020 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" event={"ID":"40e73073-ca8f-4948-b620-ac5e25b50c28","Type":"ContainerStarted","Data":"b6f80f1f63c76848fe3e012793e3abdf0920b5f1aa12b48e2f0078bd37466554"} Jul 08 17:20:36.075394 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.075366 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" event={"ID":"2130c551-3d33-4a3a-982f-58795364bbad","Type":"ContainerStarted","Data":"3f1b29bef6b0c8f755c3d2aa7b09c45e8570fb3ad12aea63e84c0bc7cb5203b9"} Jul 08 17:20:36.075523 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.075425 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" event={"ID":"2130c551-3d33-4a3a-982f-58795364bbad","Type":"ContainerStarted","Data":"58d11799e3d61a2f165ed4c5aabfc1750a2c454f4800a9be566e021a73646a22"} Jul 08 17:20:36.075523 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.075453 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" event={"ID":"2130c551-3d33-4a3a-982f-58795364bbad","Type":"ContainerStarted","Data":"d37ea812d370c9113d1cece7526209e7179803239ec167f73190457e08eb8b8c"} Jul 08 17:20:36.091639 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.090811 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-f9lcr" podStartSLOduration=66.811123846 podStartE2EDuration="1m8.090790215s" totalImagesPullingTime="1.279666369s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:36.090517724 +0000 UTC m=+69.022329444" watchObservedRunningTime="2026-07-08 17:20:36.090790215 +0000 UTC m=+69.022602139" Jul 08 17:20:36.170444 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.170345 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:20:36.344948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.344568 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:20:36.347508 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:36.347457 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9789d50d_762c_454d_9ac4_948c2822be04.slice/crio-6cbb7e86cb08d89bbc92df317213abd9b0fae21d153a72b68fd3b0070cdf7e7c WatchSource:0}: Error finding container 6cbb7e86cb08d89bbc92df317213abd9b0fae21d153a72b68fd3b0070cdf7e7c: Status 404 returned error can't find the container with id 6cbb7e86cb08d89bbc92df317213abd9b0fae21d153a72b68fd3b0070cdf7e7c Jul 08 17:20:36.555007 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.554929 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:36.557734 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.557709 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/90cd18a2-d4f5-440c-86b5-85922498bf45-node-exporter-tls\") pod \"node-exporter-jdjtz\" (UID: \"90cd18a2-d4f5-440c-86b5-85922498bf45\") " pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:36.594944 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:36.594909 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-jdjtz" Jul 08 17:20:37.019666 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:37.019627 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4rpfv" Jul 08 17:20:37.082579 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:37.082451 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"6cbb7e86cb08d89bbc92df317213abd9b0fae21d153a72b68fd3b0070cdf7e7c"} Jul 08 17:20:38.863947 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.863914 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:20:38.867447 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.867425 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.877922 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.877896 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 17:20:38.879347 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.879325 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:20:38.976199 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976162 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976219 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976269 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976297 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976337 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:38.976628 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:38.976378 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4x7p\" (UniqueName: \"kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077225 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077193 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077246 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077286 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077314 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077385 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.077403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.077404 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4x7p\" (UniqueName: \"kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.078508 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.078428 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.078508 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.078428 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.078721 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.078643 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.078721 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.078692 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.080836 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.080814 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.081003 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.080983 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.088683 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.088654 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-dd76f7ff9-4zl25"] Jul 08 17:20:39.091564 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.091543 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4x7p\" (UniqueName: \"kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p\") pod \"console-5b85fb5f5-w2gzj\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.093151 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.093132 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.096117 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096096 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 08 17:20:39.096227 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096096 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 08 17:20:39.096288 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096269 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-rrgxc\"" Jul 08 17:20:39.096351 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096319 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 08 17:20:39.096351 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096335 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-8ghnids0lkdom\"" Jul 08 17:20:39.096720 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.096665 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 08 17:20:39.100998 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.100950 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-dd76f7ff9-4zl25"] Jul 08 17:20:39.178456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178365 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-client-certs\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178414 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2dd6cbc6-b836-482a-a842-20eded8a399c-audit-log\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178703 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178465 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178703 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178542 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-tls\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178703 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178578 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-metrics-server-audit-profiles\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178703 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178610 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:39.178703 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178686 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-client-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.178958 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.178719 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4ptc\" (UniqueName: \"kubernetes.io/projected/2dd6cbc6-b836-482a-a842-20eded8a399c-kube-api-access-b4ptc\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279604 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279567 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-client-certs\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279796 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279615 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2dd6cbc6-b836-482a-a842-20eded8a399c-audit-log\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279796 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279652 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279796 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279776 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-tls\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279953 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279845 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-metrics-server-audit-profiles\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279953 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279898 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-client-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.279953 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.279928 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b4ptc\" (UniqueName: \"kubernetes.io/projected/2dd6cbc6-b836-482a-a842-20eded8a399c-kube-api-access-b4ptc\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.280275 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.280218 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/2dd6cbc6-b836-482a-a842-20eded8a399c-audit-log\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.280844 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.280797 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.280963 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.280869 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/2dd6cbc6-b836-482a-a842-20eded8a399c-metrics-server-audit-profiles\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.282439 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.282415 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-client-certs\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.282684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.282641 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-secret-metrics-server-tls\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.282684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.282679 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/2dd6cbc6-b836-482a-a842-20eded8a399c-client-ca-bundle\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.288370 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.288350 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4ptc\" (UniqueName: \"kubernetes.io/projected/2dd6cbc6-b836-482a-a842-20eded8a399c-kube-api-access-b4ptc\") pod \"metrics-server-dd76f7ff9-4zl25\" (UID: \"2dd6cbc6-b836-482a-a842-20eded8a399c\") " pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.408856 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.408817 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:39.510978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.510905 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm"] Jul 08 17:20:39.513802 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.513786 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.516933 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.516915 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-sxc5r\"" Jul 08 17:20:39.517196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.517179 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 08 17:20:39.526069 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.525666 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm"] Jul 08 17:20:39.617606 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.617575 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:39.617745 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.617656 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:39.622462 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.622442 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:39.683195 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.683155 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2k56k\" (UniqueName: \"kubernetes.io/projected/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-kube-api-access-2k56k\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.683357 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.683328 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.785142 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.784898 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.785142 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.784966 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2k56k\" (UniqueName: \"kubernetes.io/projected/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-kube-api-access-2k56k\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.785142 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:39.785114 2551 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 08 17:20:39.785412 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:20:39.785207 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert podName:690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d nodeName:}" failed. No retries permitted until 2026-07-08 17:20:40.285185337 +0000 UTC m=+73.216997040 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert") pod "monitoring-plugin-d5dcff7cf-7tvvm" (UID: "690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d") : secret "monitoring-plugin-cert" not found Jul 08 17:20:39.795931 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:39.795899 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2k56k\" (UniqueName: \"kubernetes.io/projected/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-kube-api-access-2k56k\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:39.834778 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:39.834738 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90cd18a2_d4f5_440c_86b5_85922498bf45.slice/crio-2a9b3770c5012dac9e4b830eaf028debdfd4a3a3448dd3d4b6f39695603ffa7c WatchSource:0}: Error finding container 2a9b3770c5012dac9e4b830eaf028debdfd4a3a3448dd3d4b6f39695603ffa7c: Status 404 returned error can't find the container with id 2a9b3770c5012dac9e4b830eaf028debdfd4a3a3448dd3d4b6f39695603ffa7c Jul 08 17:20:40.029983 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.029952 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:20:40.070637 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.070607 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-dd76f7ff9-4zl25"] Jul 08 17:20:40.076315 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:40.076285 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dd6cbc6_b836_482a_a842_20eded8a399c.slice/crio-8e5aa068e923f249f5d45c28e091dc87ac218bf228a7580818d7fbe5e7b32643 WatchSource:0}: Error finding container 8e5aa068e923f249f5d45c28e091dc87ac218bf228a7580818d7fbe5e7b32643: Status 404 returned error can't find the container with id 8e5aa068e923f249f5d45c28e091dc87ac218bf228a7580818d7fbe5e7b32643 Jul 08 17:20:40.103860 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.103668 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff" exitCode=0 Jul 08 17:20:40.103860 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.103761 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff"} Jul 08 17:20:40.112821 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.112792 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdjtz" event={"ID":"90cd18a2-d4f5-440c-86b5-85922498bf45","Type":"ContainerStarted","Data":"2a9b3770c5012dac9e4b830eaf028debdfd4a3a3448dd3d4b6f39695603ffa7c"} Jul 08 17:20:40.118175 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.117553 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wmtbj" event={"ID":"b71226d0-4ca5-4a86-acf8-95b991a31aec","Type":"ContainerStarted","Data":"f2998bc52cf95f289bc99d4081532262372cb1e4b8dd772b2a23c1538f0c9697"} Jul 08 17:20:40.118175 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.118107 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:20:40.121749 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.121683 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" event={"ID":"2130c551-3d33-4a3a-982f-58795364bbad","Type":"ContainerStarted","Data":"fb17ecb79d16bdd64a2cdab7d296ff667bd637338b3a9ef5f8bdf08b241fd233"} Jul 08 17:20:40.123715 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.123693 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85fb5f5-w2gzj" event={"ID":"06e3d86b-7904-46d7-a46a-631400a45593","Type":"ContainerStarted","Data":"7392a37a9508210106a743d7a9c20b51d8dd69697c60dd00d0f5da0c94818f3e"} Jul 08 17:20:40.125022 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.125003 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-9lxj9" event={"ID":"38b897f5-aac6-4cb3-b2d2-605d3570f32e","Type":"ContainerStarted","Data":"538ae8b330e68a603b05af10d23917281a2f764f2899ed089b89a5855a374b4e"} Jul 08 17:20:40.126932 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.126913 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" event={"ID":"40e73073-ca8f-4948-b620-ac5e25b50c28","Type":"ContainerStarted","Data":"4ea6c3c2817db6fa1ed737885534c943271945b8b0b07a2776d80a987ca5c2bd"} Jul 08 17:20:40.127005 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.126952 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" event={"ID":"40e73073-ca8f-4948-b620-ac5e25b50c28","Type":"ContainerStarted","Data":"182e94473988f7471871804d0db2873c3a5b3bb45e758e8d7ec1e054866824ab"} Jul 08 17:20:40.128907 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.128884 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" event={"ID":"2dd6cbc6-b836-482a-a842-20eded8a399c","Type":"ContainerStarted","Data":"8e5aa068e923f249f5d45c28e091dc87ac218bf228a7580818d7fbe5e7b32643"} Jul 08 17:20:40.132505 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.132466 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:20:40.166547 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.166469 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wmtbj" podStartSLOduration=66.50278981 podStartE2EDuration="1m12.166451482s" totalImagesPullingTime="5.663661672s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:40.165338502 +0000 UTC m=+73.097150225" watchObservedRunningTime="2026-07-08 17:20:40.166451482 +0000 UTC m=+73.098263205" Jul 08 17:20:40.185172 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.185118 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-54cf96dfcf-qq27s" podStartSLOduration=2.5827926789999998 podStartE2EDuration="6.185101746s" totalImagesPullingTime="3.602309067s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:40.183929148 +0000 UTC m=+73.115740871" watchObservedRunningTime="2026-07-08 17:20:40.185101746 +0000 UTC m=+73.116913465" Jul 08 17:20:40.203345 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.203298 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-9lxj9" podStartSLOduration=66.239188994 podStartE2EDuration="1m12.203281531s" totalImagesPullingTime="5.964092537s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 17:19:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:40.20188854 +0000 UTC m=+73.133700264" watchObservedRunningTime="2026-07-08 17:20:40.203281531 +0000 UTC m=+73.135093253" Jul 08 17:20:40.302193 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.300359 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:40.303713 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.303661 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d-monitoring-plugin-cert\") pod \"monitoring-plugin-d5dcff7cf-7tvvm\" (UID: \"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d\") " pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:40.429890 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.429851 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:40.581850 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.581795 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm"] Jul 08 17:20:40.812238 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:40.812204 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod690e3a58_6e9e_4bcc_83fc_c4ea7d226a8d.slice/crio-7b35442f1ba6390dbb0a376c36d3c8402ae384f636b7de454546207eacc85c6d WatchSource:0}: Error finding container 7b35442f1ba6390dbb0a376c36d3c8402ae384f636b7de454546207eacc85c6d: Status 404 returned error can't find the container with id 7b35442f1ba6390dbb0a376c36d3c8402ae384f636b7de454546207eacc85c6d Jul 08 17:20:40.999902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:40.998422 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:20:41.025437 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.025396 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:20:41.025620 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.025548 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.028841 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.028823 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 17:20:41.030858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.030839 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b25ms\"" Jul 08 17:20:41.031410 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.030924 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 17:20:41.031544 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.031525 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 17:20:41.031689 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.031446 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 17:20:41.031867 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.031849 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 17:20:41.032052 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.032035 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 17:20:41.032203 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.032189 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cbrsaahlj6an3\"" Jul 08 17:20:41.032448 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.032433 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 17:20:41.032744 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.032725 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 17:20:41.033054 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.033037 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 17:20:41.033781 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.033652 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 17:20:41.033781 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.033686 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 17:20:41.034603 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.034581 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 17:20:41.036196 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.036180 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 17:20:41.038690 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.038671 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 17:20:41.135241 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.135102 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85fb5f5-w2gzj" event={"ID":"06e3d86b-7904-46d7-a46a-631400a45593","Type":"ContainerStarted","Data":"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd"} Jul 08 17:20:41.138264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.138201 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" event={"ID":"40e73073-ca8f-4948-b620-ac5e25b50c28","Type":"ContainerStarted","Data":"dc2155dc2b906fa46cd18f99560c3c133553577917152093d54cba62321e6129"} Jul 08 17:20:41.142281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.141609 2551 generic.go:356] "Generic (PLEG): container finished" podID="90cd18a2-d4f5-440c-86b5-85922498bf45" containerID="c2c61a78f3df85baeb39e084258c4e0ec54daf1ac6eadf51cadb8375e3e00219" exitCode=0 Jul 08 17:20:41.142281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.141685 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdjtz" event={"ID":"90cd18a2-d4f5-440c-86b5-85922498bf45","Type":"ContainerDied","Data":"c2c61a78f3df85baeb39e084258c4e0ec54daf1ac6eadf51cadb8375e3e00219"} Jul 08 17:20:41.144236 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.144172 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" event={"ID":"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d","Type":"ContainerStarted","Data":"7b35442f1ba6390dbb0a376c36d3c8402ae384f636b7de454546207eacc85c6d"} Jul 08 17:20:41.156831 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.156293 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5b85fb5f5-w2gzj" podStartSLOduration=3.1562736 podStartE2EDuration="3.1562736s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:41.154196136 +0000 UTC m=+74.086007861" watchObservedRunningTime="2026-07-08 17:20:41.1562736 +0000 UTC m=+74.088085325" Jul 08 17:20:41.179088 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.178821 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7c9dbb8b7d-t4n7f" podStartSLOduration=3.219190137 podStartE2EDuration="7.178799257s" totalImagesPullingTime="3.95960912s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:41.175569727 +0000 UTC m=+74.107381450" watchObservedRunningTime="2026-07-08 17:20:41.178799257 +0000 UTC m=+74.110610981" Jul 08 17:20:41.207805 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207670 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.207805 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207715 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.207805 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207742 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.207805 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207819 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207878 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207905 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207931 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdhrt\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207962 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.207984 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208069 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208102 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208133 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208163 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208187 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208242 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208276 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208305 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208354 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.208513 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.208377 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.308922 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.308895 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309053 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.308930 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309053 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.308975 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309053 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309026 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309053 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309050 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309251 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309169 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309314 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309267 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309316 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309494 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309450 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309569 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309529 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309713 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309693 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309749 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309772 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309798 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309824 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309850 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309875 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309896 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xdhrt\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.309948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309933 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.310248 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.309953 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.310803 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.310426 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.311543 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.312165 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.313218 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.313573 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.313797 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.313944 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.314079 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.314249 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.314385 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.314886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.314846 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.319117 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.318885 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.319117 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.318938 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.319257 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.319129 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.320092 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.319756 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.320092 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.320000 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.320092 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.320014 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.321138 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.321117 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.321876 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.321839 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdhrt\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.323678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.323639 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.341531 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.341303 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:41.544469 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:41.544350 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:20:41.992062 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:20:41.992024 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd8e79c_5095_45f0_9ca5_c3b85436e7b4.slice/crio-6701152d0754cae44b84f7d25bf128797b6f83547895bd305b38e6a300425c5d WatchSource:0}: Error finding container 6701152d0754cae44b84f7d25bf128797b6f83547895bd305b38e6a300425c5d: Status 404 returned error can't find the container with id 6701152d0754cae44b84f7d25bf128797b6f83547895bd305b38e6a300425c5d Jul 08 17:20:42.150617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.150583 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" exitCode=0 Jul 08 17:20:42.151012 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.150699 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} Jul 08 17:20:42.151012 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.150813 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"6701152d0754cae44b84f7d25bf128797b6f83547895bd305b38e6a300425c5d"} Jul 08 17:20:42.153283 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.153254 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdjtz" event={"ID":"90cd18a2-d4f5-440c-86b5-85922498bf45","Type":"ContainerStarted","Data":"c1308d75ab6966a2834270c93750f09a1daa55940b3b9e506e2f8409bee15dc7"} Jul 08 17:20:42.153372 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.153314 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-jdjtz" event={"ID":"90cd18a2-d4f5-440c-86b5-85922498bf45","Type":"ContainerStarted","Data":"f80c4c56b73f1c331a49fb1657ce9e4f1cc5fd6d0e89f4e23186dced87384272"} Jul 08 17:20:42.204448 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:42.204387 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-jdjtz" podStartSLOduration=7.042333708 podStartE2EDuration="8.204367149s" totalImagesPullingTime="1.034173327s" totalInitContainerRuntime="127.860114ms" isStatefulPod=true podCreationTimestamp="2026-07-08 17:20:34 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:42.204091063 +0000 UTC m=+75.135902817" watchObservedRunningTime="2026-07-08 17:20:42.204367149 +0000 UTC m=+75.136178874" Jul 08 17:20:44.167115 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.167075 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246"} Jul 08 17:20:44.167562 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.167182 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224"} Jul 08 17:20:44.167562 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.167229 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4"} Jul 08 17:20:44.167562 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.167272 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6"} Jul 08 17:20:44.167562 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.167309 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574"} Jul 08 17:20:44.168544 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.168513 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" event={"ID":"690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d","Type":"ContainerStarted","Data":"36649dcc44d7f27263c7f00edc385ca936773a5e034064576b7623dcc999c8de"} Jul 08 17:20:44.168724 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.168705 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:44.170311 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.170283 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" event={"ID":"2dd6cbc6-b836-482a-a842-20eded8a399c","Type":"ContainerStarted","Data":"e3978a83bc65966fa5bb1df9d11611ff4eedf5c4fba63423c058059fc5205e95"} Jul 08 17:20:44.173858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.173836 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" Jul 08 17:20:44.185290 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.185241 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-d5dcff7cf-7tvvm" podStartSLOduration=2.832836553 podStartE2EDuration="5.185228247s" totalImagesPullingTime="2.352391694s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:44.184245566 +0000 UTC m=+77.116057300" watchObservedRunningTime="2026-07-08 17:20:44.185228247 +0000 UTC m=+77.117039971" Jul 08 17:20:44.201664 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:44.201625 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" podStartSLOduration=2.124327548 podStartE2EDuration="5.201607609s" totalImagesPullingTime="3.077280061s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:39 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:44.201109396 +0000 UTC m=+77.132921118" watchObservedRunningTime="2026-07-08 17:20:44.201607609 +0000 UTC m=+77.133419331" Jul 08 17:20:45.180396 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:45.180357 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerStarted","Data":"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94"} Jul 08 17:20:45.210675 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:45.210623 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.469997015 podStartE2EDuration="10.210604134s" totalImagesPullingTime="7.71184835s" totalInitContainerRuntime="28.758769ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:35 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:45.207818041 +0000 UTC m=+78.139629779" watchObservedRunningTime="2026-07-08 17:20:45.210604134 +0000 UTC m=+78.142415856" Jul 08 17:20:45.484815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:45.484730 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:20:46.197108 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:46.196609 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} Jul 08 17:20:47.202719 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:47.202684 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} Jul 08 17:20:48.213189 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:48.213156 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} Jul 08 17:20:48.213602 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:48.213240 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} Jul 08 17:20:48.213602 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:48.213281 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} Jul 08 17:20:49.179252 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:49.179127 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:20:49.220590 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:49.220554 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerStarted","Data":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} Jul 08 17:20:49.257358 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:49.257294 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.610000281 podStartE2EDuration="9.257272116s" totalImagesPullingTime="5.627382845s" totalInitContainerRuntime="19.88899ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:20:40 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:20:49.255625429 +0000 UTC m=+82.187437152" watchObservedRunningTime="2026-07-08 17:20:49.257272116 +0000 UTC m=+82.189083839" Jul 08 17:20:51.341687 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:51.341651 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:20:56.106801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:56.106765 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:20:59.409437 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:59.409391 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:20:59.409963 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:20:59.409466 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:21:10.507547 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.507468 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5b85fb5f5-w2gzj" podUID="06e3d86b-7904-46d7-a46a-631400a45593" containerName="console" containerID="cri-o://5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd" gracePeriod=15 Jul 08 17:21:10.751995 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.751975 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:21:10.884096 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884063 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884108 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884147 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b4x7p\" (UniqueName: \"kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884172 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884221 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884261 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884573 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884286 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config\") pod \"06e3d86b-7904-46d7-a46a-631400a45593\" (UID: \"06e3d86b-7904-46d7-a46a-631400a45593\") " Jul 08 17:21:10.884793 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884732 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:10.884892 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884689 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:10.884892 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884794 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca" (OuterVolumeSpecName: "service-ca") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:10.884892 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.884859 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config" (OuterVolumeSpecName: "console-config") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:10.886552 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.886527 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:10.886732 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.886714 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:10.886831 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.886812 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p" (OuterVolumeSpecName: "kube-api-access-b4x7p") pod "06e3d86b-7904-46d7-a46a-631400a45593" (UID: "06e3d86b-7904-46d7-a46a-631400a45593"). InnerVolumeSpecName "kube-api-access-b4x7p". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:10.985334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985296 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985329 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b4x7p\" (UniqueName: \"kubernetes.io/projected/06e3d86b-7904-46d7-a46a-631400a45593-kube-api-access-b4x7p\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985342 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-trusted-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985598 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985356 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-oauth-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985598 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985370 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-service-ca\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985598 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985382 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/06e3d86b-7904-46d7-a46a-631400a45593-console-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:10.985598 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:10.985392 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/06e3d86b-7904-46d7-a46a-631400a45593-console-oauth-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:11.307458 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.307428 2551 generic.go:356] "Generic (PLEG): container finished" podID="06e3d86b-7904-46d7-a46a-631400a45593" containerID="5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd" exitCode=0 Jul 08 17:21:11.307623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.307510 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5b85fb5f5-w2gzj" Jul 08 17:21:11.307623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.307510 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85fb5f5-w2gzj" event={"ID":"06e3d86b-7904-46d7-a46a-631400a45593","Type":"ContainerDied","Data":"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd"} Jul 08 17:21:11.307623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.307594 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5b85fb5f5-w2gzj" event={"ID":"06e3d86b-7904-46d7-a46a-631400a45593","Type":"ContainerDied","Data":"7392a37a9508210106a743d7a9c20b51d8dd69697c60dd00d0f5da0c94818f3e"} Jul 08 17:21:11.307623 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.307614 2551 scope.go:117] "RemoveContainer" containerID="5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd" Jul 08 17:21:11.317163 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.317144 2551 scope.go:117] "RemoveContainer" containerID="5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd" Jul 08 17:21:11.317428 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:11.317410 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd\": container with ID starting with 5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd not found: ID does not exist" containerID="5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd" Jul 08 17:21:11.317558 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.317436 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd"} err="failed to get container status \"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd\": rpc error: code = NotFound desc = could not find container \"5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd\": container with ID starting with 5414cf5643c2ae3f598fd48deacac937dbb235e94ce265702920d40730069dbd not found: ID does not exist" Jul 08 17:21:11.330303 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.330280 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:21:11.334033 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.334010 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5b85fb5f5-w2gzj"] Jul 08 17:21:11.720297 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:11.720203 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="06e3d86b-7904-46d7-a46a-631400a45593" path="/var/lib/kubelet/pods/06e3d86b-7904-46d7-a46a-631400a45593/volumes" Jul 08 17:21:12.137184 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:12.137157 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z2hmv_26cb2c9d-dc86-4938-916d-8151673d9e04/serve-healthcheck-canary/0.log" Jul 08 17:21:12.156014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:12.155992 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wmtbj" Jul 08 17:21:19.414171 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:19.414141 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:21:19.418053 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:19.418031 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-dd76f7ff9-4zl25" Jul 08 17:21:21.126804 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.126764 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5c99cb68c4-mdb5k" podUID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" containerName="console" containerID="cri-o://a195c35c30dce1a4e85bd29450893469987f63e58c6800d04769ddf9faa7e5fd" gracePeriod=15 Jul 08 17:21:21.348117 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.348087 2551 generic.go:356] "Generic (PLEG): container finished" podID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" containerID="a195c35c30dce1a4e85bd29450893469987f63e58c6800d04769ddf9faa7e5fd" exitCode=0 Jul 08 17:21:21.348310 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.348160 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c99cb68c4-mdb5k" event={"ID":"7820f4da-bfbc-43c5-bbfe-fb53c6678c72","Type":"ContainerDied","Data":"a195c35c30dce1a4e85bd29450893469987f63e58c6800d04769ddf9faa7e5fd"} Jul 08 17:21:21.389561 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.389537 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:21:21.581910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.581882 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.581910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.581914 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.582114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.581986 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.582114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582004 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdpfr\" (UniqueName: \"kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.582114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582042 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.582114 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582084 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert\") pod \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\" (UID: \"7820f4da-bfbc-43c5-bbfe-fb53c6678c72\") " Jul 08 17:21:21.582376 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582339 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca" (OuterVolumeSpecName: "service-ca") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:21.582461 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582390 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config" (OuterVolumeSpecName: "console-config") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:21.582622 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.582596 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:21.584244 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.584210 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:21.584343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.584302 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:21.584343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.584300 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr" (OuterVolumeSpecName: "kube-api-access-xdpfr") pod "7820f4da-bfbc-43c5-bbfe-fb53c6678c72" (UID: "7820f4da-bfbc-43c5-bbfe-fb53c6678c72"). InnerVolumeSpecName "kube-api-access-xdpfr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.682975 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-service-ca\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.683008 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.683018 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-oauth-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.683027 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdpfr\" (UniqueName: \"kubernetes.io/projected/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-kube-api-access-xdpfr\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.683036 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-console-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:21.683066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:21.683045 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7820f4da-bfbc-43c5-bbfe-fb53c6678c72-oauth-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:22.354270 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:22.354236 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5c99cb68c4-mdb5k" event={"ID":"7820f4da-bfbc-43c5-bbfe-fb53c6678c72","Type":"ContainerDied","Data":"1a2bbb525d5b42c53bb72207652a2700e3ffc1880f2ac1c9017840925e59fd5c"} Jul 08 17:21:22.354768 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:22.354275 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5c99cb68c4-mdb5k" Jul 08 17:21:22.354768 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:22.354304 2551 scope.go:117] "RemoveContainer" containerID="a195c35c30dce1a4e85bd29450893469987f63e58c6800d04769ddf9faa7e5fd" Jul 08 17:21:22.373523 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:22.373497 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:21:22.378193 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:22.378160 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5c99cb68c4-mdb5k"] Jul 08 17:21:23.724888 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:23.724850 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" path="/var/lib/kubelet/pods/7820f4da-bfbc-43c5-bbfe-fb53c6678c72/volumes" Jul 08 17:21:41.342448 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:41.342408 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:21:41.363691 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:41.363662 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:21:41.446675 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:41.446649 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:21:55.061233 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061201 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:55.061815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061542 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="alertmanager" containerID="cri-o://d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" gracePeriod=120 Jul 08 17:21:55.061815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061643 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-metric" containerID="cri-o://44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" gracePeriod=120 Jul 08 17:21:55.061815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061685 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="prom-label-proxy" containerID="cri-o://1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" gracePeriod=120 Jul 08 17:21:55.061815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061740 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy" containerID="cri-o://c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" gracePeriod=120 Jul 08 17:21:55.062045 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061842 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="config-reloader" containerID="cri-o://dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" gracePeriod=120 Jul 08 17:21:55.062045 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.061662 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-web" containerID="cri-o://081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" gracePeriod=120 Jul 08 17:21:55.484740 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484651 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" exitCode=0 Jul 08 17:21:55.484740 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484679 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" exitCode=0 Jul 08 17:21:55.484740 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484688 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" exitCode=0 Jul 08 17:21:55.484740 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484693 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" exitCode=0 Jul 08 17:21:55.484740 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484722 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94"} Jul 08 17:21:55.485055 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484832 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224"} Jul 08 17:21:55.485055 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484877 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6"} Jul 08 17:21:55.485055 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:55.484916 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574"} Jul 08 17:21:56.302617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.302594 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.367790 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367760 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.367969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367820 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.367969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367843 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.367969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367865 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.367969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367887 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.367969 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.367918 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368148 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rcnnh\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368215 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368338 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368246 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368338 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368239 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:21:56.368338 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368303 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368338 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368320 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:56.368555 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368351 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368555 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368378 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.368555 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.368421 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric\") pod \"9789d50d-762c-454d-9ac4-948c2822be04\" (UID: \"9789d50d-762c-454d-9ac4-948c2822be04\") " Jul 08 17:21:56.369123 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.369099 2551 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-metrics-client-ca\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.369245 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.369126 2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-main-db\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.369709 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.369680 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:56.370877 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.370856 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:56.371019 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.370947 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.371745 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.371708 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out" (OuterVolumeSpecName: "config-out") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:21:56.371745 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.371724 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh" (OuterVolumeSpecName: "kube-api-access-rcnnh") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "kube-api-access-rcnnh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:56.373160 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.373054 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume" (OuterVolumeSpecName: "config-volume") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.373318 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.373298 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.373418 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.373399 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.374692 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.374660 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.376992 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.376959 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.382994 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.382971 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config" (OuterVolumeSpecName: "web-config") pod "9789d50d-762c-454d-9ac4-948c2822be04" (UID: "9789d50d-762c-454d-9ac4-948c2822be04"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469697 2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/9789d50d-762c-454d-9ac4-948c2822be04-config-out\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469728 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-main-tls\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469740 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469750 2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-tls-assets\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469759 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rcnnh\" (UniqueName: \"kubernetes.io/projected/9789d50d-762c-454d-9ac4-948c2822be04-kube-api-access-rcnnh\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469767 2551 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9789d50d-762c-454d-9ac4-948c2822be04-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.469778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469776 2551 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-cluster-tls-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.470095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469786 2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-web-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.470095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469794 2551 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-config-volume\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.470095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469802 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.470095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.469810 2551 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/9789d50d-762c-454d-9ac4-948c2822be04-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:56.491659 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491629 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" exitCode=0 Jul 08 17:21:56.491659 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491654 2551 generic.go:356] "Generic (PLEG): container finished" podID="9789d50d-762c-454d-9ac4-948c2822be04" containerID="081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" exitCode=0 Jul 08 17:21:56.491855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491700 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.491855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491705 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246"} Jul 08 17:21:56.491855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491777 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4"} Jul 08 17:21:56.491855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491805 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"9789d50d-762c-454d-9ac4-948c2822be04","Type":"ContainerDied","Data":"6cbb7e86cb08d89bbc92df317213abd9b0fae21d153a72b68fd3b0070cdf7e7c"} Jul 08 17:21:56.491855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.491836 2551 scope.go:117] "RemoveContainer" containerID="1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" Jul 08 17:21:56.500886 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.500505 2551 scope.go:117] "RemoveContainer" containerID="44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" Jul 08 17:21:56.508049 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.508031 2551 scope.go:117] "RemoveContainer" containerID="c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" Jul 08 17:21:56.514861 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.514843 2551 scope.go:117] "RemoveContainer" containerID="081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" Jul 08 17:21:56.518339 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.518318 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:56.522678 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.522655 2551 scope.go:117] "RemoveContainer" containerID="dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" Jul 08 17:21:56.523952 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.523934 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:56.529846 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.529831 2551 scope.go:117] "RemoveContainer" containerID="d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" Jul 08 17:21:56.536791 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.536772 2551 scope.go:117] "RemoveContainer" containerID="a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff" Jul 08 17:21:56.543433 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.543416 2551 scope.go:117] "RemoveContainer" containerID="1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" Jul 08 17:21:56.543756 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.543737 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94\": container with ID starting with 1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94 not found: ID does not exist" containerID="1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" Jul 08 17:21:56.543813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.543764 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94"} err="failed to get container status \"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94\": rpc error: code = NotFound desc = could not find container \"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94\": container with ID starting with 1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94 not found: ID does not exist" Jul 08 17:21:56.543813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.543786 2551 scope.go:117] "RemoveContainer" containerID="44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" Jul 08 17:21:56.544046 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.544028 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246\": container with ID starting with 44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246 not found: ID does not exist" containerID="44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" Jul 08 17:21:56.544110 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544055 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246"} err="failed to get container status \"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246\": rpc error: code = NotFound desc = could not find container \"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246\": container with ID starting with 44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246 not found: ID does not exist" Jul 08 17:21:56.544110 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544079 2551 scope.go:117] "RemoveContainer" containerID="c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" Jul 08 17:21:56.544373 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.544352 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224\": container with ID starting with c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224 not found: ID does not exist" containerID="c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" Jul 08 17:21:56.544441 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544380 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224"} err="failed to get container status \"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224\": rpc error: code = NotFound desc = could not find container \"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224\": container with ID starting with c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224 not found: ID does not exist" Jul 08 17:21:56.544441 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544414 2551 scope.go:117] "RemoveContainer" containerID="081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" Jul 08 17:21:56.544706 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.544689 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4\": container with ID starting with 081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4 not found: ID does not exist" containerID="081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" Jul 08 17:21:56.544759 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544710 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4"} err="failed to get container status \"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4\": rpc error: code = NotFound desc = could not find container \"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4\": container with ID starting with 081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4 not found: ID does not exist" Jul 08 17:21:56.544759 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544727 2551 scope.go:117] "RemoveContainer" containerID="dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" Jul 08 17:21:56.544944 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.544919 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6\": container with ID starting with dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6 not found: ID does not exist" containerID="dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" Jul 08 17:21:56.545012 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544941 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6"} err="failed to get container status \"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6\": rpc error: code = NotFound desc = could not find container \"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6\": container with ID starting with dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6 not found: ID does not exist" Jul 08 17:21:56.545012 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.544972 2551 scope.go:117] "RemoveContainer" containerID="d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" Jul 08 17:21:56.545207 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.545191 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574\": container with ID starting with d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574 not found: ID does not exist" containerID="d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" Jul 08 17:21:56.545244 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545214 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574"} err="failed to get container status \"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574\": rpc error: code = NotFound desc = could not find container \"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574\": container with ID starting with d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574 not found: ID does not exist" Jul 08 17:21:56.545244 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545227 2551 scope.go:117] "RemoveContainer" containerID="a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff" Jul 08 17:21:56.545438 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:21:56.545421 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff\": container with ID starting with a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff not found: ID does not exist" containerID="a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff" Jul 08 17:21:56.545501 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545446 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff"} err="failed to get container status \"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff\": rpc error: code = NotFound desc = could not find container \"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff\": container with ID starting with a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff not found: ID does not exist" Jul 08 17:21:56.545501 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545462 2551 scope.go:117] "RemoveContainer" containerID="1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94" Jul 08 17:21:56.545735 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545705 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94"} err="failed to get container status \"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94\": rpc error: code = NotFound desc = could not find container \"1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94\": container with ID starting with 1be137134e51a34e5720a0f14c75d99bc2abd2d3054efbf8c7495da4324ada94 not found: ID does not exist" Jul 08 17:21:56.545735 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545735 2551 scope.go:117] "RemoveContainer" containerID="44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246" Jul 08 17:21:56.545964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545947 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246"} err="failed to get container status \"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246\": rpc error: code = NotFound desc = could not find container \"44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246\": container with ID starting with 44d5e57dcb22fbc326a38edaa385a5fae79f9afb8ab198d883b1142257463246 not found: ID does not exist" Jul 08 17:21:56.546009 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.545965 2551 scope.go:117] "RemoveContainer" containerID="c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224" Jul 08 17:21:56.546167 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546148 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224"} err="failed to get container status \"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224\": rpc error: code = NotFound desc = could not find container \"c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224\": container with ID starting with c04c915a1a1ed48eab3a25ecb83ca7cb9d52cb4227037b9ef8f42821dc166224 not found: ID does not exist" Jul 08 17:21:56.546224 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546167 2551 scope.go:117] "RemoveContainer" containerID="081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4" Jul 08 17:21:56.546378 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546363 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4"} err="failed to get container status \"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4\": rpc error: code = NotFound desc = could not find container \"081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4\": container with ID starting with 081691698f414147967e8508c462139f3f32dfe2327675eab3bdec5b4412c0c4 not found: ID does not exist" Jul 08 17:21:56.546420 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546388 2551 scope.go:117] "RemoveContainer" containerID="dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6" Jul 08 17:21:56.546600 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546582 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6"} err="failed to get container status \"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6\": rpc error: code = NotFound desc = could not find container \"dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6\": container with ID starting with dbf5900a729ffa2e1e5a0e419c97bff8ebc3549fd98e916dcc29afaa5373a6a6 not found: ID does not exist" Jul 08 17:21:56.546653 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546601 2551 scope.go:117] "RemoveContainer" containerID="d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574" Jul 08 17:21:56.546797 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546782 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574"} err="failed to get container status \"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574\": rpc error: code = NotFound desc = could not find container \"d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574\": container with ID starting with d16e0a21051dfaa2bfaa76fa48fa2360a4b97d86422ec6cb0d020e64d8308574 not found: ID does not exist" Jul 08 17:21:56.546840 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546798 2551 scope.go:117] "RemoveContainer" containerID="a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff" Jul 08 17:21:56.546979 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.546964 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff"} err="failed to get container status \"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff\": rpc error: code = NotFound desc = could not find container \"a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff\": container with ID starting with a8b11ce541872d00ae8b51ef36cd2fd87a43ffc878864c163dc49f87388b76ff not found: ID does not exist" Jul 08 17:21:56.553124 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553101 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:56.553430 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553414 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-metric" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553433 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-metric" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553450 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="06e3d86b-7904-46d7-a46a-631400a45593" containerName="console" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553459 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="06e3d86b-7904-46d7-a46a-631400a45593" containerName="console" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553491 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="prom-label-proxy" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553501 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="prom-label-proxy" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553510 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" containerName="console" Jul 08 17:21:56.553520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553519 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" containerName="console" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553546 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553556 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553570 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="init-config-reloader" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553579 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="init-config-reloader" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553589 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="config-reloader" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553597 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="config-reloader" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553607 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="alertmanager" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553616 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="alertmanager" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553627 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-web" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553635 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-web" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553705 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="06e3d86b-7904-46d7-a46a-631400a45593" containerName="console" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553717 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553726 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="prom-label-proxy" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553737 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-metric" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553745 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="config-reloader" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553756 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="alertmanager" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553765 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9789d50d-762c-454d-9ac4-948c2822be04" containerName="kube-rbac-proxy-web" Jul 08 17:21:56.553858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.553775 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="7820f4da-bfbc-43c5-bbfe-fb53c6678c72" containerName="console" Jul 08 17:21:56.558887 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.558865 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.561942 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.561921 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 08 17:21:56.562051 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.561950 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 08 17:21:56.562051 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.561923 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 08 17:21:56.562051 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.561921 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-5bkkz\"" Jul 08 17:21:56.562288 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.562275 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 08 17:21:56.562435 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.562415 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 08 17:21:56.562566 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.562417 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 08 17:21:56.562566 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.562418 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 08 17:21:56.562763 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.562736 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 08 17:21:56.567960 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.567933 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 08 17:21:56.578603 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.578582 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:56.671438 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671393 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671438 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671435 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw7ws\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-kube-api-access-qw7ws\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671460 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671561 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671594 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-out\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671611 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-web-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671628 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671667 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671852 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671684 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671852 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671725 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671852 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671767 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671852 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671787 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.671852 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.671822 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772154 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772188 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772209 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772239 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772259 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw7ws\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-kube-api-access-qw7ws\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772276 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772323 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772355 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-out\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772378 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-web-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772404 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772453 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772497 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.772578 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772524 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.773021 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.772992 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.773529 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.773373 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.773529 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.773453 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.775852 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.775880 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-volume\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776089 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-tls-assets\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776170 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776243 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776264 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776356 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776317 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.776989 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.776973 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-web-config\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.777078 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.777063 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-config-out\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.781689 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.781671 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw7ws\" (UniqueName: \"kubernetes.io/projected/ff5b640a-52b9-4d67-84a9-9737a9f37aa8-kube-api-access-qw7ws\") pod \"alertmanager-main-0\" (UID: \"ff5b640a-52b9-4d67-84a9-9737a9f37aa8\") " pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:56.868810 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:56.868767 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 08 17:21:57.042450 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:57.042403 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 08 17:21:57.044359 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:21:57.044335 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff5b640a_52b9_4d67_84a9_9737a9f37aa8.slice/crio-3394df37a6fa9ff522ebf9984fc637f3cd2f4d08eff0f8bbaf7fd637e9d7b3f8 WatchSource:0}: Error finding container 3394df37a6fa9ff522ebf9984fc637f3cd2f4d08eff0f8bbaf7fd637e9d7b3f8: Status 404 returned error can't find the container with id 3394df37a6fa9ff522ebf9984fc637f3cd2f4d08eff0f8bbaf7fd637e9d7b3f8 Jul 08 17:21:57.497172 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:57.497135 2551 generic.go:356] "Generic (PLEG): container finished" podID="ff5b640a-52b9-4d67-84a9-9737a9f37aa8" containerID="0082938a0bfd619d9b3323dcd353b7df58ec5b3bbf4cb3881e486da05c96036b" exitCode=0 Jul 08 17:21:57.497646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:57.497234 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerDied","Data":"0082938a0bfd619d9b3323dcd353b7df58ec5b3bbf4cb3881e486da05c96036b"} Jul 08 17:21:57.497646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:57.497304 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"3394df37a6fa9ff522ebf9984fc637f3cd2f4d08eff0f8bbaf7fd637e9d7b3f8"} Jul 08 17:21:57.721523 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:57.721494 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9789d50d-762c-454d-9ac4-948c2822be04" path="/var/lib/kubelet/pods/9789d50d-762c-454d-9ac4-948c2822be04/volumes" Jul 08 17:21:58.506556 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506519 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"84dd3c5d0f3f1f3b9b222732e015f6c59706b3dc65299b72145091ab176c3fbb"} Jul 08 17:21:58.506900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506589 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"5b3f28ca9b7fcda6b89af75d395438fa3f86859f1f2ad1cbd8fe3047f8fbb603"} Jul 08 17:21:58.506900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506641 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"c687ee6f3ea8bd62291e266c4867d65caf66e596bf0bfaaf9c32d62d0ebeff68"} Jul 08 17:21:58.506900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506668 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"307ccf4c9719a45fd99fdb42747b71da41acfb2adf44b1bd8e73f23c3b7522e1"} Jul 08 17:21:58.506900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506698 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"c427efb1ccdcf252096c06b7a756b02d48f1851eb71cdf087d57d2748ab4da56"} Jul 08 17:21:58.506900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.506731 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"ff5b640a-52b9-4d67-84a9-9737a9f37aa8","Type":"ContainerStarted","Data":"0b114a9d679793843a9187f6d0787573f7cb1a31faade0f372ee572ea775eea0"} Jul 08 17:21:58.537834 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:58.537777 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.523073265 podStartE2EDuration="2.537756061s" totalImagesPullingTime="0s" totalInitContainerRuntime="14.682796ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:21:56 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:21:58.534898118 +0000 UTC m=+151.466709861" watchObservedRunningTime="2026-07-08 17:21:58.537756061 +0000 UTC m=+151.469567788" Jul 08 17:21:59.044260 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.044205 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts"] Jul 08 17:21:59.047039 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.047015 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.050952 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.050923 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 08 17:21:59.051116 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.051068 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 08 17:21:59.051248 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.051190 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 08 17:21:59.051370 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.051350 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 08 17:21:59.051881 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.051714 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 08 17:21:59.051977 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.051923 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-vn52d\"" Jul 08 17:21:59.058291 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.058238 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 08 17:21:59.079542 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.079489 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts"] Jul 08 17:21:59.195980 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.195932 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-serving-certs-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196167 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.195999 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-federate-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196167 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196032 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196167 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196067 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6mbnt\" (UniqueName: \"kubernetes.io/projected/7a4bedb4-fd1d-4f74-b61c-76c62a527219-kube-api-access-6mbnt\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196289 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196160 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196289 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196231 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196289 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196267 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-metrics-client-ca\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.196408 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.196294 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.296998 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297092 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297139 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-metrics-client-ca\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297174 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297215 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-serving-certs-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297250 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-federate-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297277 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.297341 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.297324 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6mbnt\" (UniqueName: \"kubernetes.io/projected/7a4bedb4-fd1d-4f74-b61c-76c62a527219-kube-api-access-6mbnt\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.298137 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.298091 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-metrics-client-ca\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.298137 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.298111 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-serving-certs-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.298544 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.298525 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-trusted-ca-bundle\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.300020 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.299990 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.300020 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.300011 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-federate-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.300185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.300169 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.300319 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.300297 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/7a4bedb4-fd1d-4f74-b61c-76c62a527219-telemeter-client-tls\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.313978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.313956 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6mbnt\" (UniqueName: \"kubernetes.io/projected/7a4bedb4-fd1d-4f74-b61c-76c62a527219-kube-api-access-6mbnt\") pod \"telemeter-client-5cc66bbbf8-9r4ts\" (UID: \"7a4bedb4-fd1d-4f74-b61c-76c62a527219\") " pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.358240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.358207 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" Jul 08 17:21:59.425676 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.425642 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426017 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="prometheus" containerID="cri-o://9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" gracePeriod=600 Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426442 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-thanos" containerID="cri-o://e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" gracePeriod=600 Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426583 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy" containerID="cri-o://b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" gracePeriod=600 Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426665 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-web" containerID="cri-o://6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" gracePeriod=600 Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426743 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="thanos-sidecar" containerID="cri-o://95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" gracePeriod=600 Jul 08 17:21:59.426993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.426811 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-monitoring/prometheus-k8s-0" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="config-reloader" containerID="cri-o://313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" gracePeriod=600 Jul 08 17:21:59.527020 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.526991 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts"] Jul 08 17:21:59.531692 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:21:59.531665 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7a4bedb4_fd1d_4f74_b61c_76c62a527219.slice/crio-4a8304718f339df940215d2130ef202182da5033d365b2056fff0660a9b360d9 WatchSource:0}: Error finding container 4a8304718f339df940215d2130ef202182da5033d365b2056fff0660a9b360d9: Status 404 returned error can't find the container with id 4a8304718f339df940215d2130ef202182da5033d365b2056fff0660a9b360d9 Jul 08 17:21:59.680876 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.680845 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:21:59.802179 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802089 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802179 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802128 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802179 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802153 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802179 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802211 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802228 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802263 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xdhrt\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802292 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802324 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802352 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802385 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802460 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802406 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802471 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802544 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802602 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802632 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802668 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802707 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802742 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.802921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.802773 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\" (UID: \"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4\") " Jul 08 17:21:59.803707 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.803531 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-0") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-0". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.803825 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.803805 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca" (OuterVolumeSpecName: "configmap-metrics-client-ca") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "configmap-metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.804710 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.804035 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle" (OuterVolumeSpecName: "configmap-serving-certs-ca-bundle") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "configmap-serving-certs-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.804710 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.804437 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle" (OuterVolumeSpecName: "prometheus-trusted-ca-bundle") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "prometheus-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.805159 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.805111 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out" (OuterVolumeSpecName: "config-out") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:21:59.805392 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.805244 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-1") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-1". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.805392 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.805351 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2" (OuterVolumeSpecName: "prometheus-k8s-rulefiles-2") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "prometheus-k8s-rulefiles-2". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.805392 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.805315 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle" (OuterVolumeSpecName: "configmap-kubelet-serving-ca-bundle") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "configmap-kubelet-serving-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:21:59.805692 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.805657 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db" (OuterVolumeSpecName: "prometheus-k8s-db") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "prometheus-k8s-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:21:59.807170 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807141 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy" (OuterVolumeSpecName: "secret-kube-rbac-proxy") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807170 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807162 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-thanos-sidecar-tls") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-prometheus-k8s-thanos-sidecar-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807312 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807225 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-prometheus-k8s-kube-rbac-proxy-web") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-prometheus-k8s-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807312 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807230 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:59.807312 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807242 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls" (OuterVolumeSpecName: "secret-prometheus-k8s-tls") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-prometheus-k8s-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807660 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807631 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls" (OuterVolumeSpecName: "secret-grpc-tls") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-grpc-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807790 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807771 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config" (OuterVolumeSpecName: "config") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.807854 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.807792 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt" (OuterVolumeSpecName: "kube-api-access-xdhrt") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "kube-api-access-xdhrt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:21:59.808036 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.808013 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs" (OuterVolumeSpecName: "secret-metrics-client-certs") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "secret-metrics-client-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.808116 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.808052 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file" (OuterVolumeSpecName: "thanos-prometheus-http-client-file") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "thanos-prometheus-http-client-file". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.816992 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.816966 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config" (OuterVolumeSpecName: "web-config") pod "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" (UID: "bbd8e79c-5095-45f0-9ca5-c3b85436e7b4"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:21:59.903661 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903611 2551 reconciler_common.go:299] "Volume detached for volume \"config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903661 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903652 2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-2\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903661 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903662 2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-db\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903672 2551 reconciler_common.go:299] "Volume detached for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-serving-certs-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903682 2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-trusted-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903690 2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-kube-rbac-proxy-web\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903700 2551 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-web-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903709 2551 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-config-out\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903716 2551 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-tls-assets\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903724 2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-0\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903733 2551 reconciler_common.go:299] "Volume detached for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-grpc-tls\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903742 2551 reconciler_common.go:299] "Volume detached for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-kube-rbac-proxy\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903750 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xdhrt\" (UniqueName: \"kubernetes.io/projected/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-kube-api-access-xdhrt\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903758 2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-thanos-sidecar-tls\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903768 2551 reconciler_common.go:299] "Volume detached for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-metrics-client-certs\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903777 2551 reconciler_common.go:299] "Volume detached for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-thanos-prometheus-http-client-file\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903786 2551 reconciler_common.go:299] "Volume detached for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-metrics-client-ca\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903795 2551 reconciler_common.go:299] "Volume detached for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-prometheus-k8s-rulefiles-1\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903803 2551 reconciler_common.go:299] "Volume detached for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-configmap-kubelet-serving-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:21:59.903883 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:21:59.903811 2551 reconciler_common.go:299] "Volume detached for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4-secret-prometheus-k8s-tls\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:22:00.519611 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.519571 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" event={"ID":"7a4bedb4-fd1d-4f74-b61c-76c62a527219","Type":"ContainerStarted","Data":"4a8304718f339df940215d2130ef202182da5033d365b2056fff0660a9b360d9"} Jul 08 17:22:00.523300 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523263 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" exitCode=0 Jul 08 17:22:00.523300 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523292 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" exitCode=0 Jul 08 17:22:00.523300 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523301 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" exitCode=0 Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523309 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" exitCode=0 Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523316 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" exitCode=0 Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523324 2551 generic.go:356] "Generic (PLEG): container finished" podID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" exitCode=0 Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523346 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523365 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523416 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523444 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523461 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523518 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} Jul 08 17:22:00.523576 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523558 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} Jul 08 17:22:00.524031 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523600 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} Jul 08 17:22:00.524031 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.523628 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"bbd8e79c-5095-45f0-9ca5-c3b85436e7b4","Type":"ContainerDied","Data":"6701152d0754cae44b84f7d25bf128797b6f83547895bd305b38e6a300425c5d"} Jul 08 17:22:00.535459 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.535437 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.544148 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.544120 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.552708 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.552681 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:22:00.553371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.553354 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.559079 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.559038 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:22:00.564223 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.564199 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.571003 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.570983 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.578459 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.578441 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.585545 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.585463 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.585843 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.585819 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.585918 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.585854 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.585918 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.585881 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.586194 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.586176 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.586238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586201 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.586238 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586220 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.586371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586355 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:22:00.586522 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.586501 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.586580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586531 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.586580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586551 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.586758 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586741 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="thanos-sidecar" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586761 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="thanos-sidecar" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586774 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-thanos" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586784 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-thanos" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586794 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.586793 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.586816 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586809 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586822 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586799 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586868 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="prometheus" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586878 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="prometheus" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586907 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-web" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586917 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-web" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586927 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="config-reloader" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586934 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="config-reloader" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586950 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="init-config-reloader" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.586958 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="init-config-reloader" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587036 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="config-reloader" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587044 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="prometheus" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587051 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-web" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587056 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587063 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="kube-rbac-proxy-thanos" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587071 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" containerName="thanos-sidecar" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.587067 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.587086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587088 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587103 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.587348 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587379 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587401 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:22:00.587665 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587685 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.587730 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587699 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.588014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587909 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.588014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.587932 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.588154 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588137 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.588194 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588155 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.588354 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588337 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.588426 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588356 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.588574 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588558 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.588617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588574 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.588778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588752 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.588778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.588775 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.589082 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589061 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.589198 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589084 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.589360 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589340 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.589404 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589362 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.589624 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589608 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.589676 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589626 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.589839 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589822 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.589887 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.589839 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.590090 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.590063 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.590090 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.590090 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.590573 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.590557 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.590657 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.590556 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.590657 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.590655 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.591428 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.591381 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.591428 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.591411 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.593574 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.593537 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-dockercfg-b25ms\"" Jul 08 17:22:00.593674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.593544 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-grpc-tls-cbrsaahlj6an3\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.594051 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-kube-rbac-proxy-web\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.594164 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.594436 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-client-certs\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.594548 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.594605 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595340 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-rbac-proxy\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595413 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-prometheus-http-client-file\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595486 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-thanos-sidecar-tls\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595604 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-1\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595683 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"serving-certs-ca-bundle\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.595724 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls-assets-0\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.596213 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-web-config\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.596561 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-2\"" Jul 08 17:22:00.599249 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.597555 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-tls\"" Jul 08 17:22:00.600837 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.599305 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.600837 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.599330 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.600837 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.600686 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.600837 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.600708 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.601581 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.601555 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.601679 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.601582 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.602307 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.602277 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.602551 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.602313 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.602921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.602895 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.602921 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.602921 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.603241 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.603212 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-k8s-rulefiles-0\"" Jul 08 17:22:00.603646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.603431 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.603646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.603457 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.604156 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.604126 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.604156 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.604159 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.604534 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.604513 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.604613 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.604536 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.605310 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605285 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.605382 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605314 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.605382 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605359 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"prometheus-trusted-ca-bundle\"" Jul 08 17:22:00.605635 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605602 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.605696 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605639 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.605916 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605885 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.605916 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.605908 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.606179 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606153 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.606179 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606173 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.606531 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606507 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.606531 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606531 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.606801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606778 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.606801 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.606801 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.607062 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607035 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.607062 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607062 2551 scope.go:117] "RemoveContainer" containerID="e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3" Jul 08 17:22:00.607332 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607307 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3"} err="failed to get container status \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": rpc error: code = NotFound desc = could not find container \"e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3\": container with ID starting with e915357491232611ff96938b4a4966005e83928a941ae6793db74f66b6b688f3 not found: ID does not exist" Jul 08 17:22:00.607414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607332 2551 scope.go:117] "RemoveContainer" containerID="b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c" Jul 08 17:22:00.607627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607601 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c"} err="failed to get container status \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": rpc error: code = NotFound desc = could not find container \"b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c\": container with ID starting with b4c0d928ebe4ff107de3a49e8bd3cddbafeee59d61996a7449f73a8f1782946c not found: ID does not exist" Jul 08 17:22:00.607717 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607627 2551 scope.go:117] "RemoveContainer" containerID="6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98" Jul 08 17:22:00.607907 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607885 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98"} err="failed to get container status \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": rpc error: code = NotFound desc = could not find container \"6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98\": container with ID starting with 6cfd16ede2c8dbf0a9da6457f22edc6d2f20b84d8f457fcfd02516e28fd0ed98 not found: ID does not exist" Jul 08 17:22:00.608046 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.607909 2551 scope.go:117] "RemoveContainer" containerID="95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81" Jul 08 17:22:00.608512 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.608467 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81"} err="failed to get container status \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": rpc error: code = NotFound desc = could not find container \"95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81\": container with ID starting with 95a5cb9920db4b866eb8abf1e3e3075ccec6d9eb5836f16996437988e1cccc81 not found: ID does not exist" Jul 08 17:22:00.608512 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.608512 2551 scope.go:117] "RemoveContainer" containerID="313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d" Jul 08 17:22:00.608844 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.608820 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d"} err="failed to get container status \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": rpc error: code = NotFound desc = could not find container \"313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d\": container with ID starting with 313cb0dee70a674f4ce34facb0af31610072230604d02624cfbbd78e551f444d not found: ID does not exist" Jul 08 17:22:00.608952 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.608939 2551 scope.go:117] "RemoveContainer" containerID="9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a" Jul 08 17:22:00.609297 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.609269 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a"} err="failed to get container status \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": rpc error: code = NotFound desc = could not find container \"9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a\": container with ID starting with 9ef92309df3f3256d10699c83d8cb817a779e724d3280da15a953c4a8e6c7f9a not found: ID does not exist" Jul 08 17:22:00.609394 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.609298 2551 scope.go:117] "RemoveContainer" containerID="df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e" Jul 08 17:22:00.609614 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.609577 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e"} err="failed to get container status \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": rpc error: code = NotFound desc = could not find container \"df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e\": container with ID starting with df55076de3fa83d8dc59754e286b41da5bc346f38267624f1d9cde11a2515e6e not found: ID does not exist" Jul 08 17:22:00.615107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.615081 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:22:00.710665 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710631 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710665 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710678 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710701 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710770 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710792 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710815 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.710905 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710837 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-config-out\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710903 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.710980 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711025 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711052 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rlklz\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-kube-api-access-rlklz\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711103 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711145 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711140 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711162 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711194 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711243 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-web-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711269 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711292 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711306 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.711366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.711327 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812090 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812002 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812090 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812046 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rlklz\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-kube-api-access-rlklz\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812090 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812071 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812189 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812232 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812271 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812297 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-web-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812316 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812342 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812364 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812364 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812396 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812445 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812501 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812530 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812560 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812589 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812624 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812663 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-config-out\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.812718 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812688 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.813194 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812724 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.813194 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812976 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-metrics-client-ca\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.813194 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.812979 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-2\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-2\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.813340 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.813260 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-serving-certs-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.815756 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.815628 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-tls-assets\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.815878 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.815795 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-grpc-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.816096 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.816070 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-kube-rbac-proxy-web\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.816275 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.816251 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-0\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-0\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.816857 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.816830 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-thanos-sidecar-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-thanos-sidecar-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.817281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.817260 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-trusted-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.817424 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.817402 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-db\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-db\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.817990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.817963 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-k8s-rulefiles-1\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-prometheus-k8s-rulefiles-1\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.818230 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.818209 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1e25cac0-2418-4f20-af1d-bc55f9f20489-configmap-kubelet-serving-ca-bundle\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.818391 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.818366 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-prometheus-k8s-tls\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-prometheus-k8s-tls\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.818964 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.818940 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"thanos-prometheus-http-client-file\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-thanos-prometheus-http-client-file\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.819205 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.819151 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/1e25cac0-2418-4f20-af1d-bc55f9f20489-config-out\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.819359 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.819322 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-web-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.819530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.819511 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-config\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.820887 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.820857 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-client-certs\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-metrics-client-certs\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.820979 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.820908 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/1e25cac0-2418-4f20-af1d-bc55f9f20489-secret-kube-rbac-proxy\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.821338 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.821320 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rlklz\" (UniqueName: \"kubernetes.io/projected/1e25cac0-2418-4f20-af1d-bc55f9f20489-kube-api-access-rlklz\") pod \"prometheus-k8s-0\" (UID: \"1e25cac0-2418-4f20-af1d-bc55f9f20489\") " pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:00.908748 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:00.908710 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:01.065287 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:01.065246 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-k8s-0"] Jul 08 17:22:01.406399 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:22:01.406326 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e25cac0_2418_4f20_af1d_bc55f9f20489.slice/crio-6538f22dbe447d47699e36d6e1aef1c7bdd692b4909fc6257bc45c4fbec67232 WatchSource:0}: Error finding container 6538f22dbe447d47699e36d6e1aef1c7bdd692b4909fc6257bc45c4fbec67232: Status 404 returned error can't find the container with id 6538f22dbe447d47699e36d6e1aef1c7bdd692b4909fc6257bc45c4fbec67232 Jul 08 17:22:01.530235 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:01.530205 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" event={"ID":"7a4bedb4-fd1d-4f74-b61c-76c62a527219","Type":"ContainerStarted","Data":"4aca5293086cfd550754f7f2b76a695856dfafaf03033fc67cdc0380652f4840"} Jul 08 17:22:01.532254 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:01.532225 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"52248b00d81ab09d069ec62e930f1ff77a4084c847bc2d6a86d3093694ba4a07"} Jul 08 17:22:01.532378 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:01.532296 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"6538f22dbe447d47699e36d6e1aef1c7bdd692b4909fc6257bc45c4fbec67232"} Jul 08 17:22:01.720352 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:01.720317 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd8e79c-5095-45f0-9ca5-c3b85436e7b4" path="/var/lib/kubelet/pods/bbd8e79c-5095-45f0-9ca5-c3b85436e7b4/volumes" Jul 08 17:22:02.540519 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:02.540463 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" event={"ID":"7a4bedb4-fd1d-4f74-b61c-76c62a527219","Type":"ContainerStarted","Data":"b141f3b910db6a206b25af5a7e3e425e13d6a94a0bc444ba4a9958798dc1eec7"} Jul 08 17:22:02.540712 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:02.540607 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" event={"ID":"7a4bedb4-fd1d-4f74-b61c-76c62a527219","Type":"ContainerStarted","Data":"18460bb3192159281d5a13ca783444f17037d395e7e375a5bcf95be056c0784c"} Jul 08 17:22:02.542269 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:02.542229 2551 generic.go:356] "Generic (PLEG): container finished" podID="1e25cac0-2418-4f20-af1d-bc55f9f20489" containerID="52248b00d81ab09d069ec62e930f1ff77a4084c847bc2d6a86d3093694ba4a07" exitCode=0 Jul 08 17:22:02.542395 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:02.542321 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerDied","Data":"52248b00d81ab09d069ec62e930f1ff77a4084c847bc2d6a86d3093694ba4a07"} Jul 08 17:22:02.566681 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:02.566627 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-5cc66bbbf8-9r4ts" podStartSLOduration=1.67485046 podStartE2EDuration="3.566609285s" totalImagesPullingTime="1.891758825s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:21:59 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:02.564848252 +0000 UTC m=+155.496659985" watchObservedRunningTime="2026-07-08 17:22:02.566609285 +0000 UTC m=+155.498421011" Jul 08 17:22:03.421165 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.421123 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:22:03.423646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.423624 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.433279 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.433258 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 08 17:22:03.433826 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.433811 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 08 17:22:03.434365 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.434348 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 08 17:22:03.435222 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.435207 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 08 17:22:03.435378 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.435360 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 08 17:22:03.435453 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.435430 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 08 17:22:03.435585 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.435570 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-528mx\"" Jul 08 17:22:03.436072 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.436057 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 08 17:22:03.440086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.440068 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 08 17:22:03.455270 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.455238 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:22:03.538914 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.538877 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjrpv\" (UniqueName: \"kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539206 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539170 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539368 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539339 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539504 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539428 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539552 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539516 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539552 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539543 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.539634 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.539576 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.558562 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558519 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"a7045591f31ff4779233f1780b642b9405f59f237145db98fa3472cba01b5ca3"} Jul 08 17:22:03.558731 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558633 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"eb050219ce1179ceea288dfef7f388abd2e5b829073ca1c67259e0f7caaf309c"} Jul 08 17:22:03.558731 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558675 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"403f045bb91722312541915faedb52fb3689f292c238e1705b25dc892d68f10d"} Jul 08 17:22:03.558731 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558714 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"604ceffeb6c7c3585ee623ab8660be9a0620c37aa3e61f8ad144a211ba362fe5"} Jul 08 17:22:03.558909 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558753 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"f25f5a82c4d8dc2f5b6763440c32c25e8800c4b8ceeb0b2e483eaeda8a8ecd2f"} Jul 08 17:22:03.558909 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.558789 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-k8s-0" event={"ID":"1e25cac0-2418-4f20-af1d-bc55f9f20489","Type":"ContainerStarted","Data":"f3ab21ffa18f07376105b16fb33ad791628d03882dfe2b46e0382153763343cb"} Jul 08 17:22:03.593659 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.593588 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-k8s-0" podStartSLOduration=3.569449356 podStartE2EDuration="3.593564901s" totalImagesPullingTime="0s" totalInitContainerRuntime="24.115545ms" isStatefulPod=false podCreationTimestamp="2026-07-08 17:22:00 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:03.592114599 +0000 UTC m=+156.523926359" watchObservedRunningTime="2026-07-08 17:22:03.593564901 +0000 UTC m=+156.525376625" Jul 08 17:22:03.640992 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.640952 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641195 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641070 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641195 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641138 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641195 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641167 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bjrpv\" (UniqueName: \"kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641353 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641201 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641402 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641358 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641604 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641579 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641836 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641809 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.641971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.641899 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.642527 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.642500 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.642848 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.642826 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.644796 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.644768 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.645066 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.645049 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.649777 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.649753 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjrpv\" (UniqueName: \"kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv\") pod \"console-869c48ddb8-4g9ns\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.733324 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.733230 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:03.865966 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:03.865942 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:22:03.868046 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:22:03.868017 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9a83592c_42b5_4863_888d_107ac3099244.slice/crio-e4b53f55576d268d8d81579b1f7b7b7d7f85e956a326441fdcd50e66427462c9 WatchSource:0}: Error finding container e4b53f55576d268d8d81579b1f7b7b7d7f85e956a326441fdcd50e66427462c9: Status 404 returned error can't find the container with id e4b53f55576d268d8d81579b1f7b7b7d7f85e956a326441fdcd50e66427462c9 Jul 08 17:22:04.563564 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:04.563528 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869c48ddb8-4g9ns" event={"ID":"9a83592c-42b5-4863-888d-107ac3099244","Type":"ContainerStarted","Data":"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe"} Jul 08 17:22:04.564052 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:04.563586 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869c48ddb8-4g9ns" event={"ID":"9a83592c-42b5-4863-888d-107ac3099244","Type":"ContainerStarted","Data":"e4b53f55576d268d8d81579b1f7b7b7d7f85e956a326441fdcd50e66427462c9"} Jul 08 17:22:04.585223 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:04.585177 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-869c48ddb8-4g9ns" podStartSLOduration=1.585161626 podStartE2EDuration="1.585161626s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:22:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:22:04.585127362 +0000 UTC m=+157.516939086" watchObservedRunningTime="2026-07-08 17:22:04.585161626 +0000 UTC m=+157.516973345" Jul 08 17:22:05.909552 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:05.909513 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:22:13.733682 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:13.733640 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:13.733682 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:13.733692 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:13.738456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:13.738435 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:22:14.608230 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:22:14.608200 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:23:00.909205 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:00.909170 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:23:00.924354 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:00.924329 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:23:01.811047 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:01.811007 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-k8s-0" Jul 08 17:23:29.624161 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:29.624117 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:23:54.644901 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.644793 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-869c48ddb8-4g9ns" podUID="9a83592c-42b5-4863-888d-107ac3099244" containerName="console" containerID="cri-o://91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe" gracePeriod=15 Jul 08 17:23:54.889103 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.889078 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:23:54.940813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.940729 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.940813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.940772 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.940813 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.940810 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjrpv\" (UniqueName: \"kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.941031 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.940917 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.941031 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.940966 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.941031 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941013 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.941185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941038 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert\") pod \"9a83592c-42b5-4863-888d-107ac3099244\" (UID: \"9a83592c-42b5-4863-888d-107ac3099244\") " Jul 08 17:23:54.941240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941183 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config" (OuterVolumeSpecName: "console-config") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:23:54.941334 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941317 2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-console-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:54.941427 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941405 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca" (OuterVolumeSpecName: "service-ca") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:23:54.941520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941413 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:23:54.941520 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.941470 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:23:54.942984 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.942951 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv" (OuterVolumeSpecName: "kube-api-access-bjrpv") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "kube-api-access-bjrpv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:23:54.943109 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.943087 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:23:54.943213 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.943191 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "9a83592c-42b5-4863-888d-107ac3099244" (UID: "9a83592c-42b5-4863-888d-107ac3099244"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:23:54.997095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.997061 2551 generic.go:356] "Generic (PLEG): container finished" podID="9a83592c-42b5-4863-888d-107ac3099244" containerID="91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe" exitCode=0 Jul 08 17:23:54.997283 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.997106 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869c48ddb8-4g9ns" event={"ID":"9a83592c-42b5-4863-888d-107ac3099244","Type":"ContainerDied","Data":"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe"} Jul 08 17:23:54.997283 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.997114 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-869c48ddb8-4g9ns" Jul 08 17:23:54.997283 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.997144 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-869c48ddb8-4g9ns" event={"ID":"9a83592c-42b5-4863-888d-107ac3099244","Type":"ContainerDied","Data":"e4b53f55576d268d8d81579b1f7b7b7d7f85e956a326441fdcd50e66427462c9"} Jul 08 17:23:54.997283 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:54.997163 2551 scope.go:117] "RemoveContainer" containerID="91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe" Jul 08 17:23:55.005390 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.005372 2551 scope.go:117] "RemoveContainer" containerID="91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe" Jul 08 17:23:55.005678 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:23:55.005658 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe\": container with ID starting with 91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe not found: ID does not exist" containerID="91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe" Jul 08 17:23:55.005744 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.005689 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe"} err="failed to get container status \"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe\": rpc error: code = NotFound desc = could not find container \"91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe\": container with ID starting with 91f230c5a17c8c0b93526ec8d496b7aca068cb0acaf38ade83eebdb390bbd0fe not found: ID does not exist" Jul 08 17:23:55.017509 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.017471 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:23:55.021420 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.021398 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-869c48ddb8-4g9ns"] Jul 08 17:23:55.042625 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042593 2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-service-ca\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.042625 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042623 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjrpv\" (UniqueName: \"kubernetes.io/projected/9a83592c-42b5-4863-888d-107ac3099244-kube-api-access-bjrpv\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.042625 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042632 2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-trusted-ca-bundle\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.042781 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042642 2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/9a83592c-42b5-4863-888d-107ac3099244-oauth-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.042781 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042650 2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-oauth-config\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.042781 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.042661 2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/9a83592c-42b5-4863-888d-107ac3099244-console-serving-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:23:55.720253 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:23:55.720211 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9a83592c-42b5-4863-888d-107ac3099244" path="/var/lib/kubelet/pods/9a83592c-42b5-4863-888d-107ac3099244/volumes" Jul 08 17:24:27.620545 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:24:27.620514 2551 kubelet.go:1648] "Image garbage collection succeeded" Jul 08 17:25:01.288993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.288964 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c"] Jul 08 17:25:01.291293 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.289267 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9a83592c-42b5-4863-888d-107ac3099244" containerName="console" Jul 08 17:25:01.291293 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.289277 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9a83592c-42b5-4863-888d-107ac3099244" containerName="console" Jul 08 17:25:01.291293 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.289337 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="9a83592c-42b5-4863-888d-107ac3099244" containerName="console" Jul 08 17:25:01.292056 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.292041 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.294752 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.294732 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 08 17:25:01.294858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.294759 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 08 17:25:01.295889 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.295874 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-d22h7\"" Jul 08 17:25:01.305501 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.302820 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c"] Jul 08 17:25:01.391880 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.391846 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l9vvr\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-kube-api-access-l9vvr\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.392076 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.391897 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.493102 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.493063 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l9vvr\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-kube-api-access-l9vvr\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.493282 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.493131 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.503261 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.503238 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.503367 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.503339 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-l9vvr\" (UniqueName: \"kubernetes.io/projected/4bf4bab7-3d01-4ba0-baea-11a0b39e842f-kube-api-access-l9vvr\") pod \"cert-manager-cainjector-559cdd8c87-lsr7c\" (UID: \"4bf4bab7-3d01-4ba0-baea-11a0b39e842f\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.609614 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.609582 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" Jul 08 17:25:01.748237 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.748201 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c"] Jul 08 17:25:01.751386 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:25:01.751357 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4bf4bab7_3d01_4ba0_baea_11a0b39e842f.slice/crio-e8afa68ce338025138fb7741616773b64491a1d8c659d3c833cf3065fe90f070 WatchSource:0}: Error finding container e8afa68ce338025138fb7741616773b64491a1d8c659d3c833cf3065fe90f070: Status 404 returned error can't find the container with id e8afa68ce338025138fb7741616773b64491a1d8c659d3c833cf3065fe90f070 Jul 08 17:25:01.753276 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:01.753259 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:25:02.252701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:02.252663 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" event={"ID":"4bf4bab7-3d01-4ba0-baea-11a0b39e842f","Type":"ContainerStarted","Data":"e8afa68ce338025138fb7741616773b64491a1d8c659d3c833cf3065fe90f070"} Jul 08 17:25:05.267547 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:05.267505 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" event={"ID":"4bf4bab7-3d01-4ba0-baea-11a0b39e842f","Type":"ContainerStarted","Data":"a1ee75f0882050815be90d61d5b30063b2994912bb9a87e0054e5696ec240cb0"} Jul 08 17:25:05.297007 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:05.296954 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-lsr7c" podStartSLOduration=1.246352814 podStartE2EDuration="4.296937331s" totalImagesPullingTime="3.050584517s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:01 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:05.294010612 +0000 UTC m=+338.225822335" watchObservedRunningTime="2026-07-08 17:25:05.296937331 +0000 UTC m=+338.228749053" Jul 08 17:25:11.069150 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.069112 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf"] Jul 08 17:25:11.072646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.072624 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.076028 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.076011 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 08 17:25:11.076127 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.076011 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-429l4\"" Jul 08 17:25:11.077047 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.077029 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 08 17:25:11.084896 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.084873 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf"] Jul 08 17:25:11.176187 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.176149 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn8sk\" (UniqueName: \"kubernetes.io/projected/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-kube-api-access-wn8sk\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.176371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.176201 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.276879 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.276827 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.277047 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.276980 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn8sk\" (UniqueName: \"kubernetes.io/projected/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-kube-api-access-wn8sk\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.277221 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.277199 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-tmp\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.288602 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.288575 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn8sk\" (UniqueName: \"kubernetes.io/projected/c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55-kube-api-access-wn8sk\") pod \"openshift-lws-operator-fd8ccff4c-k5hcf\" (UID: \"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55\") " pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.382312 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.382215 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" Jul 08 17:25:11.510596 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:11.510570 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf"] Jul 08 17:25:11.512874 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:25:11.512848 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc6176f80_6ae3_4e8e_ab2f_7f215cf1bd55.slice/crio-7b632e8d9c82e9a3ceb5bc8bac2a3cf20d3445070b4748c877a5d0d40478159c WatchSource:0}: Error finding container 7b632e8d9c82e9a3ceb5bc8bac2a3cf20d3445070b4748c877a5d0d40478159c: Status 404 returned error can't find the container with id 7b632e8d9c82e9a3ceb5bc8bac2a3cf20d3445070b4748c877a5d0d40478159c Jul 08 17:25:12.297523 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:12.297463 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" event={"ID":"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55","Type":"ContainerStarted","Data":"7b632e8d9c82e9a3ceb5bc8bac2a3cf20d3445070b4748c877a5d0d40478159c"} Jul 08 17:25:14.308708 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:14.308673 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" event={"ID":"c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55","Type":"ContainerStarted","Data":"25e192e615269119ac124adb34b62b561296ef5ae7701ddce29b3d3e93aa53a0"} Jul 08 17:25:14.330662 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:14.330603 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-fd8ccff4c-k5hcf" podStartSLOduration=0.627999236 podStartE2EDuration="3.330581949s" totalImagesPullingTime="2.702582713s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:11 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:14.328499086 +0000 UTC m=+347.260310807" watchObservedRunningTime="2026-07-08 17:25:14.330581949 +0000 UTC m=+347.262393673" Jul 08 17:25:36.117695 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.117617 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp"] Jul 08 17:25:36.123674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.123648 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.126596 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.126571 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 08 17:25:36.126987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.126970 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 08 17:25:36.127608 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.127585 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-5mnw4\"" Jul 08 17:25:36.127719 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.127613 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 08 17:25:36.127719 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.127633 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 08 17:25:36.144056 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.144025 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp"] Jul 08 17:25:36.190355 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.190315 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.190530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.190367 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-webhook-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.190530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.190405 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-manifests\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.190530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.190449 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dm6mm\" (UniqueName: \"kubernetes.io/projected/bc48fce2-cb08-4e49-9b71-d8288319f25a-kube-api-access-dm6mm\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.190530 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.190504 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-tmp\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.291760 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.291722 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.291760 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.291764 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-webhook-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.292035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.291800 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-manifests\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.292035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.291837 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dm6mm\" (UniqueName: \"kubernetes.io/projected/bc48fce2-cb08-4e49-9b71-d8288319f25a-kube-api-access-dm6mm\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.292035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.291874 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-tmp\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.292322 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.292301 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-manifests\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.294240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.294202 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bc48fce2-cb08-4e49-9b71-d8288319f25a-tmp\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.294346 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.294275 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-webhook-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.294386 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.294375 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/bc48fce2-cb08-4e49-9b71-d8288319f25a-apiservice-cert\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.311253 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.311217 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dm6mm\" (UniqueName: \"kubernetes.io/projected/bc48fce2-cb08-4e49-9b71-d8288319f25a-kube-api-access-dm6mm\") pod \"opendatahub-operator-controller-manager-574956cb5f-9czqp\" (UID: \"bc48fce2-cb08-4e49-9b71-d8288319f25a\") " pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.434832 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.434748 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:36.574559 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:36.574527 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp"] Jul 08 17:25:36.577880 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:25:36.577849 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc48fce2_cb08_4e49_9b71_d8288319f25a.slice/crio-ad5077c128da7f2dfbcab6dcb661994b96ca94ac10a68e553f3a5e7ca632c388 WatchSource:0}: Error finding container ad5077c128da7f2dfbcab6dcb661994b96ca94ac10a68e553f3a5e7ca632c388: Status 404 returned error can't find the container with id ad5077c128da7f2dfbcab6dcb661994b96ca94ac10a68e553f3a5e7ca632c388 Jul 08 17:25:37.402397 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:37.402353 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" event={"ID":"bc48fce2-cb08-4e49-9b71-d8288319f25a","Type":"ContainerStarted","Data":"ad5077c128da7f2dfbcab6dcb661994b96ca94ac10a68e553f3a5e7ca632c388"} Jul 08 17:25:40.418585 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:40.418508 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" event={"ID":"bc48fce2-cb08-4e49-9b71-d8288319f25a","Type":"ContainerStarted","Data":"28c9216729695d5b5edc8b5bca1b8a1cc12d757def2e92d4c973db472fb1e040"} Jul 08 17:25:41.423646 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:41.423609 2551 generic.go:356] "Generic (PLEG): container finished" podID="bc48fce2-cb08-4e49-9b71-d8288319f25a" containerID="28c9216729695d5b5edc8b5bca1b8a1cc12d757def2e92d4c973db472fb1e040" exitCode=0 Jul 08 17:25:41.424024 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:41.423701 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" event={"ID":"bc48fce2-cb08-4e49-9b71-d8288319f25a","Type":"ContainerDied","Data":"28c9216729695d5b5edc8b5bca1b8a1cc12d757def2e92d4c973db472fb1e040"} Jul 08 17:25:42.430744 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:42.430645 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" event={"ID":"bc48fce2-cb08-4e49-9b71-d8288319f25a","Type":"ContainerStarted","Data":"580e20bb74e56ed94434bc970a1881ac5616f17a6380b2eb211f3b75f7008c1a"} Jul 08 17:25:42.431140 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:42.430765 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:53.437856 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:53.437823 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" Jul 08 17:25:53.462216 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:25:53.462158 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-574956cb5f-9czqp" podStartSLOduration=12.194420584 podStartE2EDuration="17.462142173s" totalImagesPullingTime="4.239567538s" totalInitContainerRuntime="1.028154051s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:25:36 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:25:42.457300835 +0000 UTC m=+375.389112570" watchObservedRunningTime="2026-07-08 17:25:53.462142173 +0000 UTC m=+386.393953895" Jul 08 17:26:03.562029 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.561993 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp"] Jul 08 17:26:03.566616 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.566593 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.569737 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.569712 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 08 17:26:03.570985 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.570962 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 08 17:26:03.570985 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.570980 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-mn2gr\"" Jul 08 17:26:03.571142 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.570985 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 08 17:26:03.577931 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.577901 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp"] Jul 08 17:26:03.632627 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.632583 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3712c867-b363-4afc-bacf-84dd0d563d3c-manager-config\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.632846 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.632680 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sqmgx\" (UniqueName: \"kubernetes.io/projected/3712c867-b363-4afc-bacf-84dd0d563d3c-kube-api-access-sqmgx\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.632846 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.632715 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.632846 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.632792 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-metrics-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.733660 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.733623 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sqmgx\" (UniqueName: \"kubernetes.io/projected/3712c867-b363-4afc-bacf-84dd0d563d3c-kube-api-access-sqmgx\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.733858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.733675 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.733858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.733701 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-metrics-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.733858 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.733726 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3712c867-b363-4afc-bacf-84dd0d563d3c-manager-config\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.734462 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.734441 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3712c867-b363-4afc-bacf-84dd0d563d3c-manager-config\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.736326 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.736302 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.736416 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.736357 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3712c867-b363-4afc-bacf-84dd0d563d3c-metrics-cert\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.744772 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.744750 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sqmgx\" (UniqueName: \"kubernetes.io/projected/3712c867-b363-4afc-bacf-84dd0d563d3c-kube-api-access-sqmgx\") pod \"lws-controller-manager-7b689d6b77-6mlbp\" (UID: \"3712c867-b363-4afc-bacf-84dd0d563d3c\") " pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:03.876329 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:03.876238 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:04.026002 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:26:04.025970 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3712c867_b363_4afc_bacf_84dd0d563d3c.slice/crio-7d2aff9f62953da5344d0c3ca37320dc880c75878cad190dc1d1b0f0cdc15d86 WatchSource:0}: Error finding container 7d2aff9f62953da5344d0c3ca37320dc880c75878cad190dc1d1b0f0cdc15d86: Status 404 returned error can't find the container with id 7d2aff9f62953da5344d0c3ca37320dc880c75878cad190dc1d1b0f0cdc15d86 Jul 08 17:26:04.027539 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:04.027516 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp"] Jul 08 17:26:04.529990 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:04.529956 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" event={"ID":"3712c867-b363-4afc-bacf-84dd0d563d3c","Type":"ContainerStarted","Data":"7d2aff9f62953da5344d0c3ca37320dc880c75878cad190dc1d1b0f0cdc15d86"} Jul 08 17:26:06.541287 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.541244 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" event={"ID":"3712c867-b363-4afc-bacf-84dd0d563d3c","Type":"ContainerStarted","Data":"f95b1578b308b8b771cfd2db614bbfee45af003e6ce52bd66a6bcd8ded5ae335"} Jul 08 17:26:06.541828 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.541319 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:26:06.573607 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.573552 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" podStartSLOduration=2.080459538 podStartE2EDuration="3.573535131s" totalImagesPullingTime="1.493075593s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:26:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:26:06.5679651 +0000 UTC m=+399.499776823" watchObservedRunningTime="2026-07-08 17:26:06.573535131 +0000 UTC m=+399.505346854" Jul 08 17:26:06.728151 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.728121 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf"] Jul 08 17:26:06.731432 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.731416 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.734926 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.734902 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 08 17:26:06.735098 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.735079 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-82xj2\"" Jul 08 17:26:06.735229 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.735215 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 08 17:26:06.735412 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.735399 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 08 17:26:06.736577 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.736560 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 08 17:26:06.745585 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.745563 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf"] Jul 08 17:26:06.862488 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.862452 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f8606c0-3e95-4ca5-89db-4af92096ec30-tmp\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.862738 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.862518 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8606c0-3e95-4ca5-89db-4af92096ec30-tls-certs\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.862738 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.862573 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cgsn\" (UniqueName: \"kubernetes.io/projected/0f8606c0-3e95-4ca5-89db-4af92096ec30-kube-api-access-7cgsn\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.963817 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.963782 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f8606c0-3e95-4ca5-89db-4af92096ec30-tmp\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.963817 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.963821 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8606c0-3e95-4ca5-89db-4af92096ec30-tls-certs\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.964061 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.963840 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cgsn\" (UniqueName: \"kubernetes.io/projected/0f8606c0-3e95-4ca5-89db-4af92096ec30-kube-api-access-7cgsn\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.966042 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.966007 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0f8606c0-3e95-4ca5-89db-4af92096ec30-tmp\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.966246 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.966220 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0f8606c0-3e95-4ca5-89db-4af92096ec30-tls-certs\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:06.977637 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:06.977614 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cgsn\" (UniqueName: \"kubernetes.io/projected/0f8606c0-3e95-4ca5-89db-4af92096ec30-kube-api-access-7cgsn\") pod \"kube-auth-proxy-7d76f9f859-x65xf\" (UID: \"0f8606c0-3e95-4ca5-89db-4af92096ec30\") " pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:07.040949 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:07.040913 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" Jul 08 17:26:07.199749 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:07.199705 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf"] Jul 08 17:26:07.202876 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:26:07.202847 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f8606c0_3e95_4ca5_89db_4af92096ec30.slice/crio-05efb142a8da3bb32fca95bdd8151e487cb4c45998d23f3f8461e713b018a36d WatchSource:0}: Error finding container 05efb142a8da3bb32fca95bdd8151e487cb4c45998d23f3f8461e713b018a36d: Status 404 returned error can't find the container with id 05efb142a8da3bb32fca95bdd8151e487cb4c45998d23f3f8461e713b018a36d Jul 08 17:26:07.551339 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:07.551248 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" event={"ID":"0f8606c0-3e95-4ca5-89db-4af92096ec30","Type":"ContainerStarted","Data":"05efb142a8da3bb32fca95bdd8151e487cb4c45998d23f3f8461e713b018a36d"} Jul 08 17:26:11.573199 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:11.573166 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" event={"ID":"0f8606c0-3e95-4ca5-89db-4af92096ec30","Type":"ContainerStarted","Data":"a600590c7ec95fe58782c82d5ce9a2e3aad968078357083cc01023597f59a942"} Jul 08 17:26:11.589273 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:11.589218 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7d76f9f859-x65xf" podStartSLOduration=2.108056989 podStartE2EDuration="5.589201886s" totalImagesPullingTime="3.481144897s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:26:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:26:11.588861985 +0000 UTC m=+404.520673720" watchObservedRunningTime="2026-07-08 17:26:11.589201886 +0000 UTC m=+404.521013609" Jul 08 17:26:17.553584 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:26:17.553544 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7b689d6b77-6mlbp" Jul 08 17:27:51.341050 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.341019 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc"] Jul 08 17:27:51.344198 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.344180 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.347240 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.347211 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 08 17:27:51.347350 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.347220 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 08 17:27:51.347350 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.347246 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 08 17:27:51.347350 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.347269 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 08 17:27:51.348551 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.348537 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-lrmw5\"" Jul 08 17:27:51.353690 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.353671 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc"] Jul 08 17:27:51.445423 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.445384 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb379b0-7ddd-4107-8289-f772c47904dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.445637 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.445456 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb379b0-7ddd-4107-8289-f772c47904dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.445637 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.445501 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nnh27\" (UniqueName: \"kubernetes.io/projected/bbb379b0-7ddd-4107-8289-f772c47904dc-kube-api-access-nnh27\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.546900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.546861 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb379b0-7ddd-4107-8289-f772c47904dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.546900 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.546901 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nnh27\" (UniqueName: \"kubernetes.io/projected/bbb379b0-7ddd-4107-8289-f772c47904dc-kube-api-access-nnh27\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.547116 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.546960 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb379b0-7ddd-4107-8289-f772c47904dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.547559 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.547530 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/bbb379b0-7ddd-4107-8289-f772c47904dc-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.549365 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.549337 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/bbb379b0-7ddd-4107-8289-f772c47904dc-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.561679 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.561650 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nnh27\" (UniqueName: \"kubernetes.io/projected/bbb379b0-7ddd-4107-8289-f772c47904dc-kube-api-access-nnh27\") pod \"kuadrant-console-plugin-66b5774985-2pptc\" (UID: \"bbb379b0-7ddd-4107-8289-f772c47904dc\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.653971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.653881 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" Jul 08 17:27:51.786116 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:51.786027 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc"] Jul 08 17:27:51.788721 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:27:51.788688 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbbb379b0_7ddd_4107_8289_f772c47904dc.slice/crio-9616d581ccbff26f289f5ba599defc762718daafb2bded0f667e095bb77f84d2 WatchSource:0}: Error finding container 9616d581ccbff26f289f5ba599defc762718daafb2bded0f667e095bb77f84d2: Status 404 returned error can't find the container with id 9616d581ccbff26f289f5ba599defc762718daafb2bded0f667e095bb77f84d2 Jul 08 17:27:52.014588 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:27:52.014505 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" event={"ID":"bbb379b0-7ddd-4107-8289-f772c47904dc","Type":"ContainerStarted","Data":"9616d581ccbff26f289f5ba599defc762718daafb2bded0f667e095bb77f84d2"} Jul 08 17:28:01.422989 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.422953 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz"] Jul 08 17:28:01.426706 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.426684 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.430175 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.430151 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-hc4qn\"" Jul 08 17:28:01.443134 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.443096 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz"] Jul 08 17:28:01.536792 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.536750 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.536993 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.536925 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qvr5n\" (UniqueName: \"kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.637825 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.637789 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qvr5n\" (UniqueName: \"kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.638014 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.637837 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.638339 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.638319 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.650206 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.650175 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qvr5n\" (UniqueName: \"kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.737422 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.737320 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:01.882698 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:01.882669 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz"] Jul 08 17:28:01.885564 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:28:01.885525 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4e7a98a9_acf7_468c_8a0b_185aaea79ff8.slice/crio-dd12ede02eac8caab30f0c98806de2d1b72bdb4f99b520ca24c421bc3ae63bc8 WatchSource:0}: Error finding container dd12ede02eac8caab30f0c98806de2d1b72bdb4f99b520ca24c421bc3ae63bc8: Status 404 returned error can't find the container with id dd12ede02eac8caab30f0c98806de2d1b72bdb4f99b520ca24c421bc3ae63bc8 Jul 08 17:28:02.043943 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.043857 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz"] Jul 08 17:28:02.052081 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.049654 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz"] Jul 08 17:28:02.066431 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.066397 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:02.071641 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.071618 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.090059 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.090024 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:02.141977 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.141928 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.142143 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.142049 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hnfv5\" (UniqueName: \"kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.243237 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.243202 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.243410 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.243282 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hnfv5\" (UniqueName: \"kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.243600 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.243581 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.255454 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.255432 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hnfv5\" (UniqueName: \"kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5\") pod \"kuadrant-operator-controller-manager-6fb7879bc9-gcwcd\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:02.384787 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:02.384749 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:16.995171 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:16.995146 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:16.997662 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:28:16.997629 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeefbbaf2_0981_4ad9_9bc3_905dd2a461d6.slice/crio-6c46dd1898f6971ba8b37968d757efa54d51bd0872db49884287d8c106d7abaf WatchSource:0}: Error finding container 6c46dd1898f6971ba8b37968d757efa54d51bd0872db49884287d8c106d7abaf: Status 404 returned error can't find the container with id 6c46dd1898f6971ba8b37968d757efa54d51bd0872db49884287d8c106d7abaf Jul 08 17:28:17.157729 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:17.157651 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" event={"ID":"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6","Type":"ContainerStarted","Data":"6c46dd1898f6971ba8b37968d757efa54d51bd0872db49884287d8c106d7abaf"} Jul 08 17:28:18.164193 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:18.164079 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" event={"ID":"bbb379b0-7ddd-4107-8289-f772c47904dc","Type":"ContainerStarted","Data":"ccc3ec5996d79a8dbf2681b7d52c42d700494b70839d817cbd0d3d70ef78df2f"} Jul 08 17:28:18.196722 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:18.196665 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-2pptc" podStartSLOduration=1.620470399 podStartE2EDuration="27.196645959s" totalImagesPullingTime="25.57617556s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:27:51 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:28:18.194725066 +0000 UTC m=+531.126536805" watchObservedRunningTime="2026-07-08 17:28:18.196645959 +0000 UTC m=+531.128457682" Jul 08 17:28:23.192088 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.192047 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" event={"ID":"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6","Type":"ContainerStarted","Data":"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff"} Jul 08 17:28:23.192701 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.192159 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:23.194369 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.194336 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" containerName="manager" containerID="cri-o://87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627" gracePeriod=2 Jul 08 17:28:23.213650 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.213605 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" podStartSLOduration=15.168810331 podStartE2EDuration="21.21358834s" totalImagesPullingTime="6.044778009s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:28:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:28:23.2115827 +0000 UTC m=+536.143394423" watchObservedRunningTime="2026-07-08 17:28:23.21358834 +0000 UTC m=+536.145400063" Jul 08 17:28:23.214349 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:23.214326 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" is forbidden: User \"system:node:ip-10-0-141-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-16.ec2.internal' and this object" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:23.435199 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.435171 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:23.437881 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:23.437850 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" is forbidden: User \"system:node:ip-10-0-141-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-16.ec2.internal' and this object" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:23.543633 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.543590 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qvr5n\" (UniqueName: \"kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n\") pod \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " Jul 08 17:28:23.543833 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.543665 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume\") pod \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\" (UID: \"4e7a98a9-acf7-468c-8a0b-185aaea79ff8\") " Jul 08 17:28:23.543916 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.543888 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "4e7a98a9-acf7-468c-8a0b-185aaea79ff8" (UID: "4e7a98a9-acf7-468c-8a0b-185aaea79ff8"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:28:23.545865 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.545832 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n" (OuterVolumeSpecName: "kube-api-access-qvr5n") pod "4e7a98a9-acf7-468c-8a0b-185aaea79ff8" (UID: "4e7a98a9-acf7-468c-8a0b-185aaea79ff8"). InnerVolumeSpecName "kube-api-access-qvr5n". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:28:23.644328 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.644234 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qvr5n\" (UniqueName: \"kubernetes.io/projected/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-kube-api-access-qvr5n\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:28:23.644328 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.644267 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/4e7a98a9-acf7-468c-8a0b-185aaea79ff8-extensions-socket-volume\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:28:23.721938 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:23.721905 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" path="/var/lib/kubelet/pods/4e7a98a9-acf7-468c-8a0b-185aaea79ff8/volumes" Jul 08 17:28:24.200184 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:24.200153 2551 generic.go:356] "Generic (PLEG): container finished" podID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" containerID="87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627" exitCode=2 Jul 08 17:28:24.200681 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:24.200202 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:24.200681 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:24.200257 2551 scope.go:117] "RemoveContainer" containerID="87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627" Jul 08 17:28:24.202897 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:24.202867 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" is forbidden: User \"system:node:ip-10-0-141-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-16.ec2.internal' and this object" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:24.205031 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:24.205007 2551 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-6fb7879bc9-smsgz\" is forbidden: User \"system:node:ip-10-0-141-16.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-141-16.ec2.internal' and this object" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-smsgz" Jul 08 17:28:24.208857 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:24.208839 2551 scope.go:117] "RemoveContainer" containerID="87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627" Jul 08 17:28:24.209137 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:24.209119 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627\": container with ID starting with 87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627 not found: ID does not exist" containerID="87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627" Jul 08 17:28:24.209185 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:24.209147 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627"} err="failed to get container status \"87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627\": rpc error: code = NotFound desc = could not find container \"87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627\": container with ID starting with 87e30efdc7ef619605da852983b9745ab3d466f273269356f65605fae70c9627 not found: ID does not exist" Jul 08 17:28:34.202948 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:34.202917 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:46.674127 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:46.674094 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:46.674575 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:46.674334 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" podUID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" containerName="manager" containerID="cri-o://0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff" gracePeriod=10 Jul 08 17:28:46.935636 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:46.935577 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:47.059897 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.059862 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume\") pod \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " Jul 08 17:28:47.060094 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.059916 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hnfv5\" (UniqueName: \"kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5\") pod \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\" (UID: \"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6\") " Jul 08 17:28:47.060312 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.060285 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" (UID: "eefbbaf2-0981-4ad9-9bc3-905dd2a461d6"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 08 17:28:47.061961 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.061936 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5" (OuterVolumeSpecName: "kube-api-access-hnfv5") pod "eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" (UID: "eefbbaf2-0981-4ad9-9bc3-905dd2a461d6"). InnerVolumeSpecName "kube-api-access-hnfv5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:28:47.160950 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.160916 2551 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-extensions-socket-volume\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:28:47.160950 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.160943 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hnfv5\" (UniqueName: \"kubernetes.io/projected/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6-kube-api-access-hnfv5\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:28:47.315026 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.314994 2551 generic.go:356] "Generic (PLEG): container finished" podID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" containerID="0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff" exitCode=0 Jul 08 17:28:47.315166 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.315031 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" event={"ID":"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6","Type":"ContainerDied","Data":"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff"} Jul 08 17:28:47.315166 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.315044 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" Jul 08 17:28:47.315166 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.315068 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd" event={"ID":"eefbbaf2-0981-4ad9-9bc3-905dd2a461d6","Type":"ContainerDied","Data":"6c46dd1898f6971ba8b37968d757efa54d51bd0872db49884287d8c106d7abaf"} Jul 08 17:28:47.315166 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.315089 2551 scope.go:117] "RemoveContainer" containerID="0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff" Jul 08 17:28:47.324805 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.324785 2551 scope.go:117] "RemoveContainer" containerID="0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff" Jul 08 17:28:47.325101 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:28:47.325081 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff\": container with ID starting with 0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff not found: ID does not exist" containerID="0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff" Jul 08 17:28:47.325176 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.325115 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff"} err="failed to get container status \"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff\": rpc error: code = NotFound desc = could not find container \"0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff\": container with ID starting with 0da90b6a4d1a944b8311cc0d17b31cf0f04e88575ac9e74e476684e01c3804ff not found: ID does not exist" Jul 08 17:28:47.340365 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.340340 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:47.345819 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.345796 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-6fb7879bc9-gcwcd"] Jul 08 17:28:47.722615 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:47.722538 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" path="/var/lib/kubelet/pods/eefbbaf2-0981-4ad9-9bc3-905dd2a461d6/volumes" Jul 08 17:28:52.704810 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.704781 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705132 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" containerName="manager" Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705144 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" containerName="manager" Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705158 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" containerName="manager" Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705163 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" containerName="manager" Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705230 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="eefbbaf2-0981-4ad9-9bc3-905dd2a461d6" containerName="manager" Jul 08 17:28:52.705264 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.705241 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="4e7a98a9-acf7-468c-8a0b-185aaea79ff8" containerName="manager" Jul 08 17:28:52.714211 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.714166 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.717232 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.717154 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 08 17:28:52.718972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.718946 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:28:52.805039 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.805007 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:28:52.809620 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.809592 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.809748 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.809730 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6fsp\" (UniqueName: \"kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.911078 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.911042 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p6fsp\" (UniqueName: \"kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.911374 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.911353 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.911910 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.911892 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:52.918978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:52.918958 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6fsp\" (UniqueName: \"kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp\") pod \"limitador-limitador-64c94fd64f-ldvj5\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:53.026471 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.026373 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:53.164570 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.164546 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:28:53.166779 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:28:53.166754 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4a17c0e_9625_445d_9f10_76cbf8882f49.slice/crio-425ff608e763f91134e4c683f98c7eecfc76d3ae91c7bba58a0b839fb1648cdf WatchSource:0}: Error finding container 425ff608e763f91134e4c683f98c7eecfc76d3ae91c7bba58a0b839fb1648cdf: Status 404 returned error can't find the container with id 425ff608e763f91134e4c683f98c7eecfc76d3ae91c7bba58a0b839fb1648cdf Jul 08 17:28:53.343209 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.343167 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" event={"ID":"c4a17c0e-9625-445d-9f10-76cbf8882f49","Type":"ContainerStarted","Data":"425ff608e763f91134e4c683f98c7eecfc76d3ae91c7bba58a0b839fb1648cdf"} Jul 08 17:28:53.705241 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.705173 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:28:53.708280 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.708264 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:28:53.711180 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.711161 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4nk7v\"" Jul 08 17:28:53.714725 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.714688 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:28:53.819659 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.819619 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzwnx\" (UniqueName: \"kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx\") pod \"authorino-75d556bd77-mmvc9\" (UID: \"d3c599f0-ab9e-4c88-882d-67d95deb15e4\") " pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:28:53.921421 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.920933 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzwnx\" (UniqueName: \"kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx\") pod \"authorino-75d556bd77-mmvc9\" (UID: \"d3c599f0-ab9e-4c88-882d-67d95deb15e4\") " pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:28:53.931123 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:53.931063 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzwnx\" (UniqueName: \"kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx\") pod \"authorino-75d556bd77-mmvc9\" (UID: \"d3c599f0-ab9e-4c88-882d-67d95deb15e4\") " pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:28:54.020086 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:54.019990 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:28:54.201326 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:54.201293 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:28:54.203449 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:28:54.203419 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd3c599f0_ab9e_4c88_882d_67d95deb15e4.slice/crio-9d738500bb963b53b23e5f71a450568e1af7f819c9017e2c0e824e9810b8ecc0 WatchSource:0}: Error finding container 9d738500bb963b53b23e5f71a450568e1af7f819c9017e2c0e824e9810b8ecc0: Status 404 returned error can't find the container with id 9d738500bb963b53b23e5f71a450568e1af7f819c9017e2c0e824e9810b8ecc0 Jul 08 17:28:54.349875 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:54.349828 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvc9" event={"ID":"d3c599f0-ab9e-4c88-882d-67d95deb15e4","Type":"ContainerStarted","Data":"9d738500bb963b53b23e5f71a450568e1af7f819c9017e2c0e824e9810b8ecc0"} Jul 08 17:28:56.362978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:56.362781 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" event={"ID":"c4a17c0e-9625-445d-9f10-76cbf8882f49","Type":"ContainerStarted","Data":"71aad50317dc536def2617207ef826d004cb8656431c862eac1c65b34a679476"} Jul 08 17:28:56.363448 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:56.363048 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:28:56.384788 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:56.384732 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" podStartSLOduration=1.605384276 podStartE2EDuration="4.384709493s" totalImagesPullingTime="2.779325217s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:28:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:28:56.382061251 +0000 UTC m=+569.313872975" watchObservedRunningTime="2026-07-08 17:28:56.384709493 +0000 UTC m=+569.316521219" Jul 08 17:28:59.379873 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:59.379830 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvc9" event={"ID":"d3c599f0-ab9e-4c88-882d-67d95deb15e4","Type":"ContainerStarted","Data":"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba"} Jul 08 17:28:59.397800 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:28:59.397756 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-75d556bd77-mmvc9" podStartSLOduration=2.214568401 podStartE2EDuration="6.39773889s" totalImagesPullingTime="4.183170489s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:28:53 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:28:59.395833394 +0000 UTC m=+572.327645117" watchObservedRunningTime="2026-07-08 17:28:59.39773889 +0000 UTC m=+572.329550613" Jul 08 17:29:07.370347 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:07.370313 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:29:07.895788 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:07.895752 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:29:07.895970 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:07.895923 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" podUID="c4a17c0e-9625-445d-9f10-76cbf8882f49" containerName="limitador" containerID="cri-o://71aad50317dc536def2617207ef826d004cb8656431c862eac1c65b34a679476" gracePeriod=30 Jul 08 17:29:08.421792 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.421762 2551 generic.go:356] "Generic (PLEG): container finished" podID="c4a17c0e-9625-445d-9f10-76cbf8882f49" containerID="71aad50317dc536def2617207ef826d004cb8656431c862eac1c65b34a679476" exitCode=0 Jul 08 17:29:08.422142 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.421814 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" event={"ID":"c4a17c0e-9625-445d-9f10-76cbf8882f49","Type":"ContainerDied","Data":"71aad50317dc536def2617207ef826d004cb8656431c862eac1c65b34a679476"} Jul 08 17:29:08.443614 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.443590 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:29:08.556617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.556536 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p6fsp\" (UniqueName: \"kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp\") pod \"c4a17c0e-9625-445d-9f10-76cbf8882f49\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " Jul 08 17:29:08.556617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.556579 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file\") pod \"c4a17c0e-9625-445d-9f10-76cbf8882f49\" (UID: \"c4a17c0e-9625-445d-9f10-76cbf8882f49\") " Jul 08 17:29:08.556960 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.556933 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file" (OuterVolumeSpecName: "config-file") pod "c4a17c0e-9625-445d-9f10-76cbf8882f49" (UID: "c4a17c0e-9625-445d-9f10-76cbf8882f49"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 08 17:29:08.558594 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.558574 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp" (OuterVolumeSpecName: "kube-api-access-p6fsp") pod "c4a17c0e-9625-445d-9f10-76cbf8882f49" (UID: "c4a17c0e-9625-445d-9f10-76cbf8882f49"). InnerVolumeSpecName "kube-api-access-p6fsp". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:29:08.657621 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.657585 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p6fsp\" (UniqueName: \"kubernetes.io/projected/c4a17c0e-9625-445d-9f10-76cbf8882f49-kube-api-access-p6fsp\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:29:08.657621 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:08.657612 2551 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/c4a17c0e-9625-445d-9f10-76cbf8882f49-config-file\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:29:09.427902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.427862 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" Jul 08 17:29:09.427902 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.427896 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-64c94fd64f-ldvj5" event={"ID":"c4a17c0e-9625-445d-9f10-76cbf8882f49","Type":"ContainerDied","Data":"425ff608e763f91134e4c683f98c7eecfc76d3ae91c7bba58a0b839fb1648cdf"} Jul 08 17:29:09.428383 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.427958 2551 scope.go:117] "RemoveContainer" containerID="71aad50317dc536def2617207ef826d004cb8656431c862eac1c65b34a679476" Jul 08 17:29:09.448911 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.448886 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:29:09.452741 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.452719 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-64c94fd64f-ldvj5"] Jul 08 17:29:09.720899 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:09.720820 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c4a17c0e-9625-445d-9f10-76cbf8882f49" path="/var/lib/kubelet/pods/c4a17c0e-9625-445d-9f10-76cbf8882f49/volumes" Jul 08 17:29:14.516778 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.516739 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs"] Jul 08 17:29:14.517259 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.517099 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c4a17c0e-9625-445d-9f10-76cbf8882f49" containerName="limitador" Jul 08 17:29:14.517259 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.517110 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c4a17c0e-9625-445d-9f10-76cbf8882f49" containerName="limitador" Jul 08 17:29:14.517259 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.517165 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="c4a17c0e-9625-445d-9f10-76cbf8882f49" containerName="limitador" Jul 08 17:29:14.526737 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.526707 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.529171 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.529142 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs"] Jul 08 17:29:14.529748 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.529729 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"postgres-creds\"" Jul 08 17:29:14.529859 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.529796 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"kube-root-ca.crt\"" Jul 08 17:29:14.531204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.531179 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"odh-ai-gateway-infra\"/\"openshift-service-ca.crt\"" Jul 08 17:29:14.531204 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.531202 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"default-dockercfg-fv62j\"" Jul 08 17:29:14.617918 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.617882 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gk929\" (UniqueName: \"kubernetes.io/projected/29e2f75f-3a54-4aee-9bbb-70b984a0771f-kube-api-access-gk929\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.618095 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.618016 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29e2f75f-3a54-4aee-9bbb-70b984a0771f-data\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.719421 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.719381 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29e2f75f-3a54-4aee-9bbb-70b984a0771f-data\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.719421 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.719428 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gk929\" (UniqueName: \"kubernetes.io/projected/29e2f75f-3a54-4aee-9bbb-70b984a0771f-kube-api-access-gk929\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.719799 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.719779 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/29e2f75f-3a54-4aee-9bbb-70b984a0771f-data\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.728669 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.728639 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gk929\" (UniqueName: \"kubernetes.io/projected/29e2f75f-3a54-4aee-9bbb-70b984a0771f-kube-api-access-gk929\") pod \"postgres-69fb4594cf-6b6hs\" (UID: \"29e2f75f-3a54-4aee-9bbb-70b984a0771f\") " pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.838704 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.838674 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:14.969903 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:14.969870 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs"] Jul 08 17:29:14.971814 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:29:14.971787 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e2f75f_3a54_4aee_9bbb_70b984a0771f.slice/crio-25589e513d0408b009b6b404a0d8c4bd484205f30a216413a89be0916122d532 WatchSource:0}: Error finding container 25589e513d0408b009b6b404a0d8c4bd484205f30a216413a89be0916122d532: Status 404 returned error can't find the container with id 25589e513d0408b009b6b404a0d8c4bd484205f30a216413a89be0916122d532 Jul 08 17:29:15.462381 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:15.462343 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" event={"ID":"29e2f75f-3a54-4aee-9bbb-70b984a0771f","Type":"ContainerStarted","Data":"25589e513d0408b009b6b404a0d8c4bd484205f30a216413a89be0916122d532"} Jul 08 17:29:20.491978 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:20.491934 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" event={"ID":"29e2f75f-3a54-4aee-9bbb-70b984a0771f","Type":"ContainerStarted","Data":"5458f3d466df5a6630d8ed04ba6e7d5764c5cd2e00c8ce658c9413f85defdb6a"} Jul 08 17:29:20.492408 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:20.491999 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:20.509437 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:20.509383 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" podStartSLOduration=1.847903698 podStartE2EDuration="6.509367667s" totalImagesPullingTime="4.661463969s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:20.508078 +0000 UTC m=+593.439889745" watchObservedRunningTime="2026-07-08 17:29:20.509367667 +0000 UTC m=+593.441179432" Jul 08 17:29:26.526565 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:26.526530 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/postgres-69fb4594cf-6b6hs" Jul 08 17:29:28.950281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:28.950242 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:29:28.950855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:28.950440 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="kuadrant-system/authorino-75d556bd77-mmvc9" podUID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" containerName="authorino" containerID="cri-o://d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba" gracePeriod=30 Jul 08 17:29:29.241549 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.241519 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:29:29.358305 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.358264 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hzwnx\" (UniqueName: \"kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx\") pod \"d3c599f0-ab9e-4c88-882d-67d95deb15e4\" (UID: \"d3c599f0-ab9e-4c88-882d-67d95deb15e4\") " Jul 08 17:29:29.360292 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.360268 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx" (OuterVolumeSpecName: "kube-api-access-hzwnx") pod "d3c599f0-ab9e-4c88-882d-67d95deb15e4" (UID: "d3c599f0-ab9e-4c88-882d-67d95deb15e4"). InnerVolumeSpecName "kube-api-access-hzwnx". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:29:29.459885 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.459797 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hzwnx\" (UniqueName: \"kubernetes.io/projected/d3c599f0-ab9e-4c88-882d-67d95deb15e4-kube-api-access-hzwnx\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:29:29.534815 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.534787 2551 generic.go:356] "Generic (PLEG): container finished" podID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" containerID="d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba" exitCode=0 Jul 08 17:29:29.534991 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.534838 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75d556bd77-mmvc9" Jul 08 17:29:29.534991 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.534879 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvc9" event={"ID":"d3c599f0-ab9e-4c88-882d-67d95deb15e4","Type":"ContainerDied","Data":"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba"} Jul 08 17:29:29.534991 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.534921 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75d556bd77-mmvc9" event={"ID":"d3c599f0-ab9e-4c88-882d-67d95deb15e4","Type":"ContainerDied","Data":"9d738500bb963b53b23e5f71a450568e1af7f819c9017e2c0e824e9810b8ecc0"} Jul 08 17:29:29.534991 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.534943 2551 scope.go:117] "RemoveContainer" containerID="d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba" Jul 08 17:29:29.543971 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.543953 2551 scope.go:117] "RemoveContainer" containerID="d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba" Jul 08 17:29:29.544263 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:29.544238 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba\": container with ID starting with d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba not found: ID does not exist" containerID="d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba" Jul 08 17:29:29.544366 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.544271 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba"} err="failed to get container status \"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba\": rpc error: code = NotFound desc = could not find container \"d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba\": container with ID starting with d2052f2ed6d759c10fb585e93974d1b8521fe4c3725a8df01e54ae856aa1c1ba not found: ID does not exist" Jul 08 17:29:29.557954 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.557918 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:29:29.562248 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.562222 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-75d556bd77-mmvc9"] Jul 08 17:29:29.722017 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:29.721925 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" path="/var/lib/kubelet/pods/d3c599f0-ab9e-4c88-882d-67d95deb15e4/volumes" Jul 08 17:29:33.442944 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.442903 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:33.443393 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.443320 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" containerName="authorino" Jul 08 17:29:33.443393 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.443336 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" containerName="authorino" Jul 08 17:29:33.443469 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.443415 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="d3c599f0-ab9e-4c88-882d-67d95deb15e4" containerName="authorino" Jul 08 17:29:33.628044 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.628007 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:33.628221 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.628112 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:33.630879 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.630846 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:29:33.631085 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.631060 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 17:29:33.631231 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.631204 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lnjqb\"" Jul 08 17:29:33.631339 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.631322 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 17:29:33.644657 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.644625 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:29:33.644856 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.644732 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.701233 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.701150 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.701233 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.701189 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:33.701233 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.701217 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rmskd\" (UniqueName: \"kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.701453 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.701320 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gndh6\" (UniqueName: \"kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:33.802525 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.802463 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.802525 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.802526 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:33.802728 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.802554 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rmskd\" (UniqueName: \"kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.802728 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.802608 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gndh6\" (UniqueName: \"kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:33.802728 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:33.802616 2551 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 08 17:29:33.802728 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:33.802678 2551 secret.go:189] Couldn't get secret opendatahub/maas-controller-webhook-cert: secret "maas-controller-webhook-cert" not found Jul 08 17:29:33.802728 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:33.802695 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert podName:65efb15e-bdfd-49eb-aeaf-bfd6198ba922 nodeName:}" failed. No retries permitted until 2026-07-08 17:29:34.302671573 +0000 UTC m=+607.234483274 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert") pod "maas-controller-5f5468d586-hqzmh" (UID: "65efb15e-bdfd-49eb-aeaf-bfd6198ba922") : secret "maas-controller-webhook-cert" not found Jul 08 17:29:33.802888 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:33.802729 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert podName:f685ef37-7b00-4110-8e94-71e23bb7f480 nodeName:}" failed. No retries permitted until 2026-07-08 17:29:34.302716162 +0000 UTC m=+607.234527863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "webhook-cert" (UniqueName: "kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert") pod "maas-controller-74bb894dcc-zdjj8" (UID: "f685ef37-7b00-4110-8e94-71e23bb7f480") : secret "maas-controller-webhook-cert" not found Jul 08 17:29:33.814503 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.814460 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rmskd\" (UniqueName: \"kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:33.814674 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:33.814608 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gndh6\" (UniqueName: \"kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:34.306758 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.306722 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:34.306940 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.306765 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:34.309157 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.309128 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") pod \"maas-controller-74bb894dcc-zdjj8\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:34.309306 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.309161 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") pod \"maas-controller-5f5468d586-hqzmh\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:34.539687 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.539647 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:34.555617 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.555590 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:34.687019 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.686993 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:34.689809 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:29:34.689782 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf685ef37_7b00_4110_8e94_71e23bb7f480.slice/crio-f636cc68966a6cb4e7786887d71f023231d505ebd38091853cc985270924cf96 WatchSource:0}: Error finding container f636cc68966a6cb4e7786887d71f023231d505ebd38091853cc985270924cf96: Status 404 returned error can't find the container with id f636cc68966a6cb4e7786887d71f023231d505ebd38091853cc985270924cf96 Jul 08 17:29:34.707989 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:34.707964 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:29:34.709758 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:29:34.709733 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod65efb15e_bdfd_49eb_aeaf_bfd6198ba922.slice/crio-cd0d84caeceadbcaf6ba85d68b5b15bc266f2904fe13e52e9f057e851da7e6b2 WatchSource:0}: Error finding container cd0d84caeceadbcaf6ba85d68b5b15bc266f2904fe13e52e9f057e851da7e6b2: Status 404 returned error can't find the container with id cd0d84caeceadbcaf6ba85d68b5b15bc266f2904fe13e52e9f057e851da7e6b2 Jul 08 17:29:35.566775 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:35.566722 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-hqzmh" event={"ID":"65efb15e-bdfd-49eb-aeaf-bfd6198ba922","Type":"ContainerStarted","Data":"cd0d84caeceadbcaf6ba85d68b5b15bc266f2904fe13e52e9f057e851da7e6b2"} Jul 08 17:29:35.568461 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:35.568431 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" event={"ID":"f685ef37-7b00-4110-8e94-71e23bb7f480","Type":"ContainerStarted","Data":"f636cc68966a6cb4e7786887d71f023231d505ebd38091853cc985270924cf96"} Jul 08 17:29:37.579236 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.579194 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-hqzmh" event={"ID":"65efb15e-bdfd-49eb-aeaf-bfd6198ba922","Type":"ContainerStarted","Data":"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a"} Jul 08 17:29:37.579757 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.579259 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:37.580715 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.580688 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" event={"ID":"f685ef37-7b00-4110-8e94-71e23bb7f480","Type":"ContainerStarted","Data":"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12"} Jul 08 17:29:37.580809 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.580740 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:37.610073 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.610024 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5468d586-hqzmh" podStartSLOduration=2.2851879 podStartE2EDuration="4.610006958s" totalImagesPullingTime="2.324819058s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:37.608341265 +0000 UTC m=+610.540152984" watchObservedRunningTime="2026-07-08 17:29:37.610006958 +0000 UTC m=+610.541818681" Jul 08 17:29:37.654995 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:37.654943 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" podStartSLOduration=2.3217140450000002 podStartE2EDuration="4.654925793s" totalImagesPullingTime="2.333211748s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:29:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:29:37.653394082 +0000 UTC m=+610.585205805" watchObservedRunningTime="2026-07-08 17:29:37.654925793 +0000 UTC m=+610.586737515" Jul 08 17:29:48.588152 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.588122 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:29:48.588547 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.588192 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:48.652006 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.651974 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:48.652202 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.652175 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" podUID="f685ef37-7b00-4110-8e94-71e23bb7f480" containerName="manager" containerID="cri-o://311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12" gracePeriod=30 Jul 08 17:29:48.901246 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.901223 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:48.938036 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.938007 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gndh6\" (UniqueName: \"kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6\") pod \"f685ef37-7b00-4110-8e94-71e23bb7f480\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " Jul 08 17:29:48.938221 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.938061 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") pod \"f685ef37-7b00-4110-8e94-71e23bb7f480\" (UID: \"f685ef37-7b00-4110-8e94-71e23bb7f480\") " Jul 08 17:29:48.940231 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.940197 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "f685ef37-7b00-4110-8e94-71e23bb7f480" (UID: "f685ef37-7b00-4110-8e94-71e23bb7f480"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:29:48.940347 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:48.940237 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6" (OuterVolumeSpecName: "kube-api-access-gndh6") pod "f685ef37-7b00-4110-8e94-71e23bb7f480" (UID: "f685ef37-7b00-4110-8e94-71e23bb7f480"). InnerVolumeSpecName "kube-api-access-gndh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:29:49.038917 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.038880 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gndh6\" (UniqueName: \"kubernetes.io/projected/f685ef37-7b00-4110-8e94-71e23bb7f480-kube-api-access-gndh6\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:29:49.038917 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.038908 2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/f685ef37-7b00-4110-8e94-71e23bb7f480-webhook-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:29:49.638638 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.638607 2551 generic.go:356] "Generic (PLEG): container finished" podID="f685ef37-7b00-4110-8e94-71e23bb7f480" containerID="311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12" exitCode=0 Jul 08 17:29:49.639107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.638650 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" Jul 08 17:29:49.639107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.638695 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" event={"ID":"f685ef37-7b00-4110-8e94-71e23bb7f480","Type":"ContainerDied","Data":"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12"} Jul 08 17:29:49.639107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.638757 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-74bb894dcc-zdjj8" event={"ID":"f685ef37-7b00-4110-8e94-71e23bb7f480","Type":"ContainerDied","Data":"f636cc68966a6cb4e7786887d71f023231d505ebd38091853cc985270924cf96"} Jul 08 17:29:49.639107 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.638812 2551 scope.go:117] "RemoveContainer" containerID="311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12" Jul 08 17:29:49.651128 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.651109 2551 scope.go:117] "RemoveContainer" containerID="311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12" Jul 08 17:29:49.651445 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:29:49.651423 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12\": container with ID starting with 311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12 not found: ID does not exist" containerID="311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12" Jul 08 17:29:49.651574 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.651450 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12"} err="failed to get container status \"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12\": rpc error: code = NotFound desc = could not find container \"311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12\": container with ID starting with 311c11df79cdc238839a66698044c4d59de505a528600b961269ba09ee38cb12 not found: ID does not exist" Jul 08 17:29:49.668172 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.668149 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:49.671798 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.671774 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-74bb894dcc-zdjj8"] Jul 08 17:29:49.721467 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:29:49.721437 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f685ef37-7b00-4110-8e94-71e23bb7f480" path="/var/lib/kubelet/pods/f685ef37-7b00-4110-8e94-71e23bb7f480/volumes" Jul 08 17:30:09.245317 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.245284 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["odh-ai-gateway-infra/maas-api-768ff58896-nrxtw"] Jul 08 17:30:09.245775 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.245678 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f685ef37-7b00-4110-8e94-71e23bb7f480" containerName="manager" Jul 08 17:30:09.245775 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.245691 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="f685ef37-7b00-4110-8e94-71e23bb7f480" containerName="manager" Jul 08 17:30:09.245775 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.245763 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="f685ef37-7b00-4110-8e94-71e23bb7f480" containerName="manager" Jul 08 17:30:09.248795 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.248773 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.251568 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.251507 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"odh-ai-gateway-infra\"/\"maas-api-serving-cert\"" Jul 08 17:30:09.262659 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.262633 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768ff58896-nrxtw"] Jul 08 17:30:09.322601 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.322562 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c748044-3db6-42e3-b34d-c34e7e08d052-openshift-service-ca\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.322776 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.322654 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7npps\" (UniqueName: \"kubernetes.io/projected/2c748044-3db6-42e3-b34d-c34e7e08d052-kube-api-access-7npps\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.322776 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.322692 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.424087 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.424052 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c748044-3db6-42e3-b34d-c34e7e08d052-openshift-service-ca\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.424261 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.424114 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7npps\" (UniqueName: \"kubernetes.io/projected/2c748044-3db6-42e3-b34d-c34e7e08d052-kube-api-access-7npps\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.424261 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.424147 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.424350 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:30:09.424261 2551 secret.go:189] Couldn't get secret odh-ai-gateway-infra/maas-api-serving-cert: secret "maas-api-serving-cert" not found Jul 08 17:30:09.424350 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:30:09.424314 2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls podName:2c748044-3db6-42e3-b34d-c34e7e08d052 nodeName:}" failed. No retries permitted until 2026-07-08 17:30:09.924298548 +0000 UTC m=+642.856110250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "maas-api-tls" (UniqueName: "kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls") pod "maas-api-768ff58896-nrxtw" (UID: "2c748044-3db6-42e3-b34d-c34e7e08d052") : secret "maas-api-serving-cert" not found Jul 08 17:30:09.424797 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.424776 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca\" (UniqueName: \"kubernetes.io/configmap/2c748044-3db6-42e3-b34d-c34e7e08d052-openshift-service-ca\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.433242 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.433215 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7npps\" (UniqueName: \"kubernetes.io/projected/2c748044-3db6-42e3-b34d-c34e7e08d052-kube-api-access-7npps\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.874212 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.874172 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-829w7"] Jul 08 17:30:09.877690 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.877675 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:09.880790 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.880758 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-jfh24\"" Jul 08 17:30:09.880790 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.880760 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 08 17:30:09.885092 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.885068 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-829w7"] Jul 08 17:30:09.905904 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.905877 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2s86k"] Jul 08 17:30:09.909227 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.909207 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:09.920311 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.920288 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2s86k"] Jul 08 17:30:09.927965 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.927938 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:09.930681 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:09.930659 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/2c748044-3db6-42e3-b34d-c34e7e08d052-maas-api-tls\") pod \"maas-api-768ff58896-nrxtw\" (UID: \"2c748044-3db6-42e3-b34d-c34e7e08d052\") " pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:10.029374 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.029339 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lbhrc\" (UniqueName: \"kubernetes.io/projected/d49c818f-9496-433a-9810-7d3bb52d939d-kube-api-access-lbhrc\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.029374 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.029379 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kv4q5\" (UniqueName: \"kubernetes.io/projected/a86ec620-a7c7-49b4-9bb4-29ede056d532-kube-api-access-kv4q5\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.029633 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.029418 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49c818f-9496-433a-9810-7d3bb52d939d-plugins-config-volume\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.029633 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.029509 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a86ec620-a7c7-49b4-9bb4-29ede056d532-plugins-config-volume\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.130580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.130460 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49c818f-9496-433a-9810-7d3bb52d939d-plugins-config-volume\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.130580 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.130543 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a86ec620-a7c7-49b4-9bb4-29ede056d532-plugins-config-volume\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.130817 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.130595 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lbhrc\" (UniqueName: \"kubernetes.io/projected/d49c818f-9496-433a-9810-7d3bb52d939d-kube-api-access-lbhrc\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.130817 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.130717 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kv4q5\" (UniqueName: \"kubernetes.io/projected/a86ec620-a7c7-49b4-9bb4-29ede056d532-kube-api-access-kv4q5\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.131282 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.131259 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/d49c818f-9496-433a-9810-7d3bb52d939d-plugins-config-volume\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.131371 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.131304 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/a86ec620-a7c7-49b4-9bb4-29ede056d532-plugins-config-volume\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.139011 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.138984 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kv4q5\" (UniqueName: \"kubernetes.io/projected/a86ec620-a7c7-49b4-9bb4-29ede056d532-kube-api-access-kv4q5\") pod \"payload-pre-processing-7964457696-829w7\" (UID: \"a86ec620-a7c7-49b4-9bb4-29ede056d532\") " pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.139199 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.139178 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lbhrc\" (UniqueName: \"kubernetes.io/projected/d49c818f-9496-433a-9810-7d3bb52d939d-kube-api-access-lbhrc\") pod \"payload-processing-58f86c7798-2s86k\" (UID: \"d49c818f-9496-433a-9810-7d3bb52d939d\") " pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.158567 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.158541 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:10.189613 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.189584 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:10.219952 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.219912 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:10.318421 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.315291 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["odh-ai-gateway-infra/maas-api-768ff58896-nrxtw"] Jul 08 17:30:10.321668 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.321646 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 17:30:10.345033 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.344994 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-7964457696-829w7"] Jul 08 17:30:10.347661 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:30:10.347630 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda86ec620_a7c7_49b4_9bb4_29ede056d532.slice/crio-3766dbd4fe523ac50c557ff7d45e9943f4dda104c5ce2fd75f25cf8cab6a3b69 WatchSource:0}: Error finding container 3766dbd4fe523ac50c557ff7d45e9943f4dda104c5ce2fd75f25cf8cab6a3b69: Status 404 returned error can't find the container with id 3766dbd4fe523ac50c557ff7d45e9943f4dda104c5ce2fd75f25cf8cab6a3b69 Jul 08 17:30:10.377987 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.377963 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-processing-58f86c7798-2s86k"] Jul 08 17:30:10.379695 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:30:10.379664 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd49c818f_9496_433a_9810_7d3bb52d939d.slice/crio-d74d15fea2d6bb5459f5678f75dcf9ce456c25ef531b5f230499c1d086a4781a WatchSource:0}: Error finding container d74d15fea2d6bb5459f5678f75dcf9ce456c25ef531b5f230499c1d086a4781a: Status 404 returned error can't find the container with id d74d15fea2d6bb5459f5678f75dcf9ce456c25ef531b5f230499c1d086a4781a Jul 08 17:30:10.739449 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.739365 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" event={"ID":"a86ec620-a7c7-49b4-9bb4-29ede056d532","Type":"ContainerStarted","Data":"3766dbd4fe523ac50c557ff7d45e9943f4dda104c5ce2fd75f25cf8cab6a3b69"} Jul 08 17:30:10.740454 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.740432 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" event={"ID":"2c748044-3db6-42e3-b34d-c34e7e08d052","Type":"ContainerStarted","Data":"1bbfe5accd0059c4c61b0dc6dede05cb3768b5723f24ba3dfc39f7b96ef1d9bb"} Jul 08 17:30:10.741641 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:10.741614 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" event={"ID":"d49c818f-9496-433a-9810-7d3bb52d939d","Type":"ContainerStarted","Data":"d74d15fea2d6bb5459f5678f75dcf9ce456c25ef531b5f230499c1d086a4781a"} Jul 08 17:30:12.756242 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.756152 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" event={"ID":"d49c818f-9496-433a-9810-7d3bb52d939d","Type":"ContainerStarted","Data":"1d0007a80257ec3a6bde406afd44f15e3884049c7f088e47faf72cbfcfacbfc2"} Jul 08 17:30:12.756242 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.756220 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:12.757609 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.757586 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" event={"ID":"a86ec620-a7c7-49b4-9bb4-29ede056d532","Type":"ContainerStarted","Data":"6302e6b5168421a14067c809453d94479d0baed44ca860472705cb6a537dac6b"} Jul 08 17:30:12.757691 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.757634 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:12.759024 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.759005 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" event={"ID":"2c748044-3db6-42e3-b34d-c34e7e08d052","Type":"ContainerStarted","Data":"77bc86e05bb334bf9d4db4ea4cdf1b63a713c01939669dd2a9a52b936d0df948"} Jul 08 17:30:12.759189 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.759172 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:12.773330 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.773288 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" podStartSLOduration=1.818924386 podStartE2EDuration="3.77327136s" totalImagesPullingTime="1.954346974s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:30:12.772169736 +0000 UTC m=+645.703981462" watchObservedRunningTime="2026-07-08 17:30:12.77327136 +0000 UTC m=+645.705083084" Jul 08 17:30:12.807847 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.807754 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" podStartSLOduration=1.82320643 podStartE2EDuration="3.80746182s" totalImagesPullingTime="1.98425539s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:30:12.789707715 +0000 UTC m=+645.721519438" watchObservedRunningTime="2026-07-08 17:30:12.80746182 +0000 UTC m=+645.739273524" Jul 08 17:30:12.813721 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:12.813662 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" podStartSLOduration=1.8016637850000001 podStartE2EDuration="3.813641343s" totalImagesPullingTime="2.011977558s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:30:09 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:30:12.811180126 +0000 UTC m=+645.742991862" watchObservedRunningTime="2026-07-08 17:30:12.813641343 +0000 UTC m=+645.745453069" Jul 08 17:30:18.770350 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:18.770318 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="odh-ai-gateway-infra/maas-api-768ff58896-nrxtw" Jul 08 17:30:23.767325 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:23.767291 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-processing-58f86c7798-2s86k" Jul 08 17:30:23.767834 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:23.767385 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-7964457696-829w7" Jul 08 17:30:32.886963 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.886929 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw"] Jul 08 17:30:32.891363 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.891344 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.894324 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.894304 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 08 17:30:32.895764 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.895745 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 08 17:30:32.895914 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.895892 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\"" Jul 08 17:30:32.896027 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.895986 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-hkglw\"" Jul 08 17:30:32.902559 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.902537 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw"] Jul 08 17:30:32.921002 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.920967 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.921284 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.921262 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-22f9q\" (UniqueName: \"kubernetes.io/projected/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-kube-api-access-22f9q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.921369 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.921353 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.921414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.921404 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.921456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.921437 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:32.921456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:32.921453 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.021935 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.021895 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.021935 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.021937 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022141 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.021974 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022141 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022015 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-22f9q\" (UniqueName: \"kubernetes.io/projected/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-kube-api-access-22f9q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022141 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022074 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022141 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022110 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022435 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022409 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-model-cache\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022514 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022460 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tmp-dir\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.022585 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.022534 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-home\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.024382 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.024361 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-dshm\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.024629 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.024611 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.030265 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.030244 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-22f9q\" (UniqueName: \"kubernetes.io/projected/be3e0b96-e66b-4d62-8b16-b5abe59dd06f-kube-api-access-22f9q\") pod \"e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw\" (UID: \"be3e0b96-e66b-4d62-8b16-b5abe59dd06f\") " pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.204059 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.203966 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:33.343445 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.343414 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw"] Jul 08 17:30:33.343928 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:30:33.343908 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe3e0b96_e66b_4d62_8b16_b5abe59dd06f.slice/crio-592b7e33a0c8b46db568c8d80c40f8a17f961671239612405ce064c594416a69 WatchSource:0}: Error finding container 592b7e33a0c8b46db568c8d80c40f8a17f961671239612405ce064c594416a69: Status 404 returned error can't find the container with id 592b7e33a0c8b46db568c8d80c40f8a17f961671239612405ce064c594416a69 Jul 08 17:30:33.868308 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:33.868261 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" event={"ID":"be3e0b96-e66b-4d62-8b16-b5abe59dd06f","Type":"ContainerStarted","Data":"592b7e33a0c8b46db568c8d80c40f8a17f961671239612405ce064c594416a69"} Jul 08 17:30:34.874689 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:34.874652 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" event={"ID":"be3e0b96-e66b-4d62-8b16-b5abe59dd06f","Type":"ContainerStarted","Data":"a9ba19e03d3fadb9e7a6624843e9a98dcedb07274b4321e6f3d4b794b65c2157"} Jul 08 17:30:34.893651 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:34.893597 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" podStartSLOduration=1.692133909 podStartE2EDuration="2.893577118s" totalImagesPullingTime="1.201443209s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:30:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:30:34.891612649 +0000 UTC m=+667.823424372" watchObservedRunningTime="2026-07-08 17:30:34.893577118 +0000 UTC m=+667.825388855" Jul 08 17:30:43.204258 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:43.204225 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:43.204769 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:43.204286 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:43.216648 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:43.216622 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:43.926260 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:43.926232 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw" Jul 08 17:30:54.091574 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.091544 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h"] Jul 08 17:30:54.096194 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.096174 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.099521 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.099468 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-trlp-test-simulated-kserve-self-signed-certs\"" Jul 08 17:30:54.105281 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.105258 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h"] Jul 08 17:30:54.223071 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223042 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401b2bfe-e862-4b6a-9652-70772f3865a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.223304 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223090 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.223304 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223122 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.223304 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223144 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.223304 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223170 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.223304 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.223188 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lm9m4\" (UniqueName: \"kubernetes.io/projected/401b2bfe-e862-4b6a-9652-70772f3865a8-kube-api-access-lm9m4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324497 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324441 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324497 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324490 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lm9m4\" (UniqueName: \"kubernetes.io/projected/401b2bfe-e862-4b6a-9652-70772f3865a8-kube-api-access-lm9m4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324546 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401b2bfe-e862-4b6a-9652-70772f3865a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324584 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324623 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324684 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324656 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.324918 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324887 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-home\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.325035 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.324927 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-model-cache\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.325078 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.325048 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-tmp-dir\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.326972 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.326948 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/401b2bfe-e862-4b6a-9652-70772f3865a8-dshm\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.327072 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.327052 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/401b2bfe-e862-4b6a-9652-70772f3865a8-tls-certs\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.332599 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.332580 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lm9m4\" (UniqueName: \"kubernetes.io/projected/401b2bfe-e862-4b6a-9652-70772f3865a8-kube-api-access-lm9m4\") pod \"e2e-trlp-test-simulated-kserve-598466548d-v4x5h\" (UID: \"401b2bfe-e862-4b6a-9652-70772f3865a8\") " pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.407744 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.407656 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:30:54.541975 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.541952 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h"] Jul 08 17:30:54.543971 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:30:54.543932 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod401b2bfe_e862_4b6a_9652_70772f3865a8.slice/crio-952d961284fb4d9f4f5fb85bf9e8e51cdcd60877f4736d0d4d7a7f2a097c2fc8 WatchSource:0}: Error finding container 952d961284fb4d9f4f5fb85bf9e8e51cdcd60877f4736d0d4d7a7f2a097c2fc8: Status 404 returned error can't find the container with id 952d961284fb4d9f4f5fb85bf9e8e51cdcd60877f4736d0d4d7a7f2a097c2fc8 Jul 08 17:30:54.967863 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.967827 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" event={"ID":"401b2bfe-e862-4b6a-9652-70772f3865a8","Type":"ContainerStarted","Data":"f658a5b2026c065cf6f6fa68d41c294a9a565837c1555c2714e49d6577e6f2b9"} Jul 08 17:30:54.968036 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.967893 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" event={"ID":"401b2bfe-e862-4b6a-9652-70772f3865a8","Type":"ContainerStarted","Data":"952d961284fb4d9f4f5fb85bf9e8e51cdcd60877f4736d0d4d7a7f2a097c2fc8"} Jul 08 17:30:54.987170 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:30:54.987078 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" podStartSLOduration=0.798825125 podStartE2EDuration="987.061555ms" totalImagesPullingTime="188.23643ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:30:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:30:54.986562161 +0000 UTC m=+687.918373884" watchObservedRunningTime="2026-07-08 17:30:54.987061555 +0000 UTC m=+687.918873277" Jul 08 17:31:04.408595 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:31:04.408557 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:31:04.408959 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:31:04.408626 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:31:04.420786 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:31:04.420763 2551 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:31:05.027577 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:31:05.027539 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-trlp-test-simulated-kserve-598466548d-v4x5h" Jul 08 17:34:01.972363 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:01.972331 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:34:01.972943 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:01.972552 2551 kuberuntime_container.go:872] "Killing container with a grace period" pod="opendatahub/maas-controller-5f5468d586-hqzmh" podUID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" containerName="manager" containerID="cri-o://2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a" gracePeriod=30 Jul 08 17:34:02.227588 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.227521 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:34:02.297221 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.297183 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rmskd\" (UniqueName: \"kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd\") pod \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " Jul 08 17:34:02.297403 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.297327 2551 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") pod \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\" (UID: \"65efb15e-bdfd-49eb-aeaf-bfd6198ba922\") " Jul 08 17:34:02.299329 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.299293 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd" (OuterVolumeSpecName: "kube-api-access-rmskd") pod "65efb15e-bdfd-49eb-aeaf-bfd6198ba922" (UID: "65efb15e-bdfd-49eb-aeaf-bfd6198ba922"). InnerVolumeSpecName "kube-api-access-rmskd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 08 17:34:02.299518 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.299331 2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert" (OuterVolumeSpecName: "webhook-cert") pod "65efb15e-bdfd-49eb-aeaf-bfd6198ba922" (UID: "65efb15e-bdfd-49eb-aeaf-bfd6198ba922"). InnerVolumeSpecName "webhook-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 08 17:34:02.398705 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.398669 2551 reconciler_common.go:299] "Volume detached for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-webhook-cert\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:34:02.398705 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.398699 2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rmskd\" (UniqueName: \"kubernetes.io/projected/65efb15e-bdfd-49eb-aeaf-bfd6198ba922-kube-api-access-rmskd\") on node \"ip-10-0-141-16.ec2.internal\" DevicePath \"\"" Jul 08 17:34:02.861157 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.861125 2551 generic.go:356] "Generic (PLEG): container finished" podID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" containerID="2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a" exitCode=0 Jul 08 17:34:02.861343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.861164 2551 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-hqzmh" Jul 08 17:34:02.861343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.861180 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-hqzmh" event={"ID":"65efb15e-bdfd-49eb-aeaf-bfd6198ba922","Type":"ContainerDied","Data":"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a"} Jul 08 17:34:02.861343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.861221 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-hqzmh" event={"ID":"65efb15e-bdfd-49eb-aeaf-bfd6198ba922","Type":"ContainerDied","Data":"cd0d84caeceadbcaf6ba85d68b5b15bc266f2904fe13e52e9f057e851da7e6b2"} Jul 08 17:34:02.861343 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.861241 2551 scope.go:117] "RemoveContainer" containerID="2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a" Jul 08 17:34:02.870650 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.870632 2551 scope.go:117] "RemoveContainer" containerID="2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a" Jul 08 17:34:02.870891 ip-10-0-141-16 kubenswrapper[2551]: E0708 17:34:02.870873 2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a\": container with ID starting with 2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a not found: ID does not exist" containerID="2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a" Jul 08 17:34:02.870935 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.870899 2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a"} err="failed to get container status \"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a\": rpc error: code = NotFound desc = could not find container \"2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a\": container with ID starting with 2eee23a0d684981df7fb04b9d64cfb938a7497c4173cffcda86b9a306fa4937a not found: ID does not exist" Jul 08 17:34:02.893983 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.893958 2551 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:34:02.906232 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:02.906208 2551 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-5f5468d586-hqzmh"] Jul 08 17:34:03.722069 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.722036 2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" path="/var/lib/kubelet/pods/65efb15e-bdfd-49eb-aeaf-bfd6198ba922/volumes" Jul 08 17:34:03.747096 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.747064 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-5f5468d586-8nlgq"] Jul 08 17:34:03.753450 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.753417 2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" containerName="manager" Jul 08 17:34:03.753450 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.753447 2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" containerName="manager" Jul 08 17:34:03.753766 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.753745 2551 memory_manager.go:368] "RemoveStaleState removing state" podUID="65efb15e-bdfd-49eb-aeaf-bfd6198ba922" containerName="manager" Jul 08 17:34:03.758693 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.758668 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.770930 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.770910 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-webhook-cert\"" Jul 08 17:34:03.772776 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.772759 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-lnjqb\"" Jul 08 17:34:03.772855 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.772814 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jul 08 17:34:03.797881 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.797847 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5468d586-8nlgq"] Jul 08 17:34:03.812306 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.812280 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c054a734-9172-404e-a6f7-bc5ee01798fd-webhook-cert\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.812400 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.812325 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g88tm\" (UniqueName: \"kubernetes.io/projected/c054a734-9172-404e-a6f7-bc5ee01798fd-kube-api-access-g88tm\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.913135 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.913098 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c054a734-9172-404e-a6f7-bc5ee01798fd-webhook-cert\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.913319 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.913144 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g88tm\" (UniqueName: \"kubernetes.io/projected/c054a734-9172-404e-a6f7-bc5ee01798fd-kube-api-access-g88tm\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.915630 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.915606 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/c054a734-9172-404e-a6f7-bc5ee01798fd-webhook-cert\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:03.926622 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:03.926600 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g88tm\" (UniqueName: \"kubernetes.io/projected/c054a734-9172-404e-a6f7-bc5ee01798fd-kube-api-access-g88tm\") pod \"maas-controller-5f5468d586-8nlgq\" (UID: \"c054a734-9172-404e-a6f7-bc5ee01798fd\") " pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:04.069440 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.069395 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:04.207414 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.207385 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-5f5468d586-8nlgq"] Jul 08 17:34:04.208898 ip-10-0-141-16 kubenswrapper[2551]: W0708 17:34:04.208868 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc054a734_9172_404e_a6f7_bc5ee01798fd.slice/crio-c7c7d9bf7ed822e6b66e7e9c8f721a4177c3685d81cef12635e40e5b16a053b2 WatchSource:0}: Error finding container c7c7d9bf7ed822e6b66e7e9c8f721a4177c3685d81cef12635e40e5b16a053b2: Status 404 returned error can't find the container with id c7c7d9bf7ed822e6b66e7e9c8f721a4177c3685d81cef12635e40e5b16a053b2 Jul 08 17:34:04.873063 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.873030 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-8nlgq" event={"ID":"c054a734-9172-404e-a6f7-bc5ee01798fd","Type":"ContainerStarted","Data":"a82a02d380d0bc2bd718767f128790864a345aac4b6f998ae4e6d753cb288039"} Jul 08 17:34:04.873457 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.873080 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-5f5468d586-8nlgq" event={"ID":"c054a734-9172-404e-a6f7-bc5ee01798fd","Type":"ContainerStarted","Data":"c7c7d9bf7ed822e6b66e7e9c8f721a4177c3685d81cef12635e40e5b16a053b2"} Jul 08 17:34:04.873457 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.873107 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 17:34:04.897844 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:04.897753 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/maas-controller-5f5468d586-8nlgq" podStartSLOduration=1.469423828 podStartE2EDuration="1.897732017s" totalImagesPullingTime="428.308189ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 17:34:03 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 17:34:04.893661275 +0000 UTC m=+877.825472997" watchObservedRunningTime="2026-07-08 17:34:04.897732017 +0000 UTC m=+877.829543740" Jul 08 17:34:15.881456 ip-10-0-141-16 kubenswrapper[2551]: I0708 17:34:15.881426 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-5f5468d586-8nlgq" Jul 08 18:05:09.170253 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:09.170224 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f5468d586-8nlgq_c054a734-9172-404e-a6f7-bc5ee01798fd/manager/0.log" Jul 08 18:05:09.651698 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:09.651674 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-574956cb5f-9czqp_bc48fce2-cb08-4e49-9b71-d8288319f25a/copy-manifests/0.log" Jul 08 18:05:09.675740 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:09.675713 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-574956cb5f-9czqp_bc48fce2-cb08-4e49-9b71-d8288319f25a/manager/0.log" Jul 08 18:05:11.570688 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:11.570662 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2pptc_bbb379b0-7ddd-4107-8289-f772c47904dc/kuadrant-console-plugin/0.log" Jul 08 18:05:12.856663 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:12.856623 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d76f9f859-x65xf_0f8606c0-3e95-4ca5-89db-4af92096ec30/kube-auth-proxy/0.log" Jul 08 18:05:13.148979 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:13.148943 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-829w7_a86ec620-a7c7-49b4-9bb4-29ede056d532/payload-pre-processing/0.log" Jul 08 18:05:13.285634 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:13.285593 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2s86k_d49c818f-9496-433a-9810-7d3bb52d939d/payload-processing/0.log" Jul 08 18:05:13.758487 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:13.758420 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-67d8d5d6d-mxjxw_be3e0b96-e66b-4d62-8b16-b5abe59dd06f/main/0.log" Jul 08 18:05:13.989696 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:13.989635 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-trlp-test-simulated-kserve-598466548d-v4x5h_401b2bfe-e862-4b6a-9652-70772f3865a8/main/0.log" Jul 08 18:05:18.439994 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.439961 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7l84t/must-gather-lqfvn"] Jul 08 18:05:18.443722 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.443704 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.447644 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.447625 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l84t\"/\"kube-root-ca.crt\"" Jul 08 18:05:18.448392 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.448377 2551 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7l84t\"/\"openshift-service-ca.crt\"" Jul 08 18:05:18.450356 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.450257 2551 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7l84t\"/\"default-dockercfg-7p7mm\"" Jul 08 18:05:18.468722 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.468691 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l84t/must-gather-lqfvn"] Jul 08 18:05:18.548845 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.548801 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/114a543b-70e1-4d46-a286-932f3d5fac89-must-gather-output\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.549026 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.548987 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4z6m5\" (UniqueName: \"kubernetes.io/projected/114a543b-70e1-4d46-a286-932f3d5fac89-kube-api-access-4z6m5\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.649738 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.649704 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4z6m5\" (UniqueName: \"kubernetes.io/projected/114a543b-70e1-4d46-a286-932f3d5fac89-kube-api-access-4z6m5\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.649738 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.649744 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/114a543b-70e1-4d46-a286-932f3d5fac89-must-gather-output\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.650024 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.650009 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/114a543b-70e1-4d46-a286-932f3d5fac89-must-gather-output\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.662291 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.662256 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4z6m5\" (UniqueName: \"kubernetes.io/projected/114a543b-70e1-4d46-a286-932f3d5fac89-kube-api-access-4z6m5\") pod \"must-gather-lqfvn\" (UID: \"114a543b-70e1-4d46-a286-932f3d5fac89\") " pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.752912 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.752826 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l84t/must-gather-lqfvn" Jul 08 18:05:18.900286 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.900245 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l84t/must-gather-lqfvn"] Jul 08 18:05:18.904842 ip-10-0-141-16 kubenswrapper[2551]: W0708 18:05:18.904815 2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod114a543b_70e1_4d46_a286_932f3d5fac89.slice/crio-50e1fe3883792e34876de5eccde6ce5d2aaf583fa6d944a2199597429be8cb98 WatchSource:0}: Error finding container 50e1fe3883792e34876de5eccde6ce5d2aaf583fa6d944a2199597429be8cb98: Status 404 returned error can't find the container with id 50e1fe3883792e34876de5eccde6ce5d2aaf583fa6d944a2199597429be8cb98 Jul 08 18:05:18.907191 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:18.907174 2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 08 18:05:19.830604 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:19.830566 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l84t/must-gather-lqfvn" event={"ID":"114a543b-70e1-4d46-a286-932f3d5fac89","Type":"ContainerStarted","Data":"50e1fe3883792e34876de5eccde6ce5d2aaf583fa6d944a2199597429be8cb98"} Jul 08 18:05:20.839843 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:20.839800 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l84t/must-gather-lqfvn" event={"ID":"114a543b-70e1-4d46-a286-932f3d5fac89","Type":"ContainerStarted","Data":"819a01a77448174b3b30c1864c3829ac98e30b06f4de6f1a1d0dc6264a2d4a16"} Jul 08 18:05:20.840359 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:20.839859 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l84t/must-gather-lqfvn" event={"ID":"114a543b-70e1-4d46-a286-932f3d5fac89","Type":"ContainerStarted","Data":"a0609d33fbc33f79fa3958293f6031349f78cedb8aeb182ac57ebb44729adb36"} Jul 08 18:05:20.862772 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:20.862709 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7l84t/must-gather-lqfvn" podStartSLOduration=1.641674945 podStartE2EDuration="2.86268384s" totalImagesPullingTime="1.221008895s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-08 18:05:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:20.859658843 +0000 UTC m=+2753.791470568" watchObservedRunningTime="2026-07-08 18:05:20.86268384 +0000 UTC m=+2753.794495565" Jul 08 18:05:21.841267 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:21.841237 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-9lxj9_38b897f5-aac6-4cb3-b2d2-605d3570f32e/global-pull-secret-syncer/0.log" Jul 08 18:05:22.051909 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:22.051879 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4fqdx_183fd0a6-e75d-459e-be3e-02ed31a476fb/konnectivity-agent/0.log" Jul 08 18:05:22.194296 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:22.194204 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-141-16.ec2.internal_c41a36ba272456960f43f5342563530e/haproxy/0.log" Jul 08 18:05:26.758637 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:26.758556 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-2pptc_bbb379b0-7ddd-4107-8289-f772c47904dc/kuadrant-console-plugin/0.log" Jul 08 18:05:28.794009 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:28.793974 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/alertmanager/0.log" Jul 08 18:05:28.852802 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:28.852699 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/config-reloader/0.log" Jul 08 18:05:28.893499 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:28.892835 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/kube-rbac-proxy-web/0.log" Jul 08 18:05:28.924605 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:28.924580 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/kube-rbac-proxy/0.log" Jul 08 18:05:28.958345 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:28.958306 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/kube-rbac-proxy-metric/0.log" Jul 08 18:05:29.000914 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.000879 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/prom-label-proxy/0.log" Jul 08 18:05:29.039726 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.039696 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_ff5b640a-52b9-4d67-84a9-9737a9f37aa8/init-config-reloader/0.log" Jul 08 18:05:29.193982 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.193951 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-t4n7f_40e73073-ca8f-4948-b620-ac5e25b50c28/kube-state-metrics/0.log" Jul 08 18:05:29.272382 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.272355 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-t4n7f_40e73073-ca8f-4948-b620-ac5e25b50c28/kube-rbac-proxy-main/0.log" Jul 08 18:05:29.353663 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.353639 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7c9dbb8b7d-t4n7f_40e73073-ca8f-4948-b620-ac5e25b50c28/kube-rbac-proxy-self/0.log" Jul 08 18:05:29.420891 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.420758 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-dd76f7ff9-4zl25_2dd6cbc6-b836-482a-a842-20eded8a399c/metrics-server/0.log" Jul 08 18:05:29.463548 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.463518 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-d5dcff7cf-7tvvm_690e3a58-6e9e-4bcc-83fc-c4ea7d226a8d/monitoring-plugin/0.log" Jul 08 18:05:29.883231 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:29.883185 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdjtz_90cd18a2-d4f5-440c-86b5-85922498bf45/node-exporter/0.log" Jul 08 18:05:30.036940 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.036846 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdjtz_90cd18a2-d4f5-440c-86b5-85922498bf45/kube-rbac-proxy/0.log" Jul 08 18:05:30.167012 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.166984 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-jdjtz_90cd18a2-d4f5-440c-86b5-85922498bf45/init-textfile/0.log" Jul 08 18:05:30.247714 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.247662 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qq27s_2130c551-3d33-4a3a-982f-58795364bbad/kube-rbac-proxy-main/0.log" Jul 08 18:05:30.294643 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.294558 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qq27s_2130c551-3d33-4a3a-982f-58795364bbad/kube-rbac-proxy-self/0.log" Jul 08 18:05:30.357393 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.357367 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-54cf96dfcf-qq27s_2130c551-3d33-4a3a-982f-58795364bbad/openshift-state-metrics/0.log" Jul 08 18:05:30.443242 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.443215 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/prometheus/0.log" Jul 08 18:05:30.473946 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.473915 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/config-reloader/0.log" Jul 08 18:05:30.524140 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.524105 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/thanos-sidecar/0.log" Jul 08 18:05:30.561648 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.561619 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/kube-rbac-proxy-web/0.log" Jul 08 18:05:30.597562 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.597520 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/kube-rbac-proxy/0.log" Jul 08 18:05:30.688739 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.688688 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/kube-rbac-proxy-thanos/0.log" Jul 08 18:05:30.759178 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.759154 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-k8s-0_1e25cac0-2418-4f20-af1d-bc55f9f20489/init-config-reloader/0.log" Jul 08 18:05:30.809811 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.809778 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-28mm4_2d03d347-1977-4785-968c-02670b56b10e/prometheus-operator/0.log" Jul 08 18:05:30.842587 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.842507 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-567f7b8b7c-28mm4_2d03d347-1977-4785-968c-02670b56b10e/kube-rbac-proxy/0.log" Jul 08 18:05:30.879747 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.879717 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-d559d66d5-54p7r_91e4425a-6440-404f-8ab7-7dfd90760ead/prometheus-operator-admission-webhook/0.log" Jul 08 18:05:30.943980 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:30.943948 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc66bbbf8-9r4ts_7a4bedb4-fd1d-4f74-b61c-76c62a527219/telemeter-client/0.log" Jul 08 18:05:31.029179 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:31.029147 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc66bbbf8-9r4ts_7a4bedb4-fd1d-4f74-b61c-76c62a527219/reload/0.log" Jul 08 18:05:31.088721 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:31.088690 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-5cc66bbbf8-9r4ts_7a4bedb4-fd1d-4f74-b61c-76c62a527219/kube-rbac-proxy/0.log" Jul 08 18:05:32.707940 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.707903 2551 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt"] Jul 08 18:05:32.715942 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.715910 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.744099 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.744060 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt"] Jul 08 18:05:32.805464 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.805428 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-proc\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.805853 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.805818 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-podres\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.806153 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.806138 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-sys\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.806327 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.806295 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvqxz\" (UniqueName: \"kubernetes.io/projected/1894a8af-c1e1-4a14-bcd5-57c34412a38c-kube-api-access-mvqxz\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.806620 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.806603 2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-lib-modules\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910617 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910581 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-sys\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910814 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910628 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mvqxz\" (UniqueName: \"kubernetes.io/projected/1894a8af-c1e1-4a14-bcd5-57c34412a38c-kube-api-access-mvqxz\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910814 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910731 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-sys\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910902 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910808 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-lib-modules\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910941 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910898 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-proc\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.910982 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910950 2551 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-podres\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.911020 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910983 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-lib-modules\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.911020 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.910995 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-proc\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.911091 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.911056 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/1894a8af-c1e1-4a14-bcd5-57c34412a38c-podres\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:32.921628 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:32.921593 2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvqxz\" (UniqueName: \"kubernetes.io/projected/1894a8af-c1e1-4a14-bcd5-57c34412a38c-kube-api-access-mvqxz\") pod \"perf-node-gather-daemonset-wl6vt\" (UID: \"1894a8af-c1e1-4a14-bcd5-57c34412a38c\") " pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:33.029262 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:33.029167 2551 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:33.225813 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:33.225612 2551 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt"] Jul 08 18:05:33.933835 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:33.933795 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" event={"ID":"1894a8af-c1e1-4a14-bcd5-57c34412a38c","Type":"ContainerStarted","Data":"b66f73370bed25c67b508254fd183a56633db08c61e008715849df2545f4e014"} Jul 08 18:05:33.934287 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:33.933863 2551 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" event={"ID":"1894a8af-c1e1-4a14-bcd5-57c34412a38c","Type":"ContainerStarted","Data":"9cc7597b653b1bf85a5eab7e39f09f1c4fef2bac01dd712269010c4595bc1497"} Jul 08 18:05:33.934287 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:33.933906 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:35.882309 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:35.882274 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rpfv_b475e0f8-649b-48ac-ba1d-c1810d8f635b/dns/0.log" Jul 08 18:05:35.912101 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:35.912070 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4rpfv_b475e0f8-649b-48ac-ba1d-c1810d8f635b/kube-rbac-proxy/0.log" Jul 08 18:05:36.110093 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:36.110064 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bbcrm_7a3a8b13-afe3-4a34-a186-1c6284e939bf/dns-node-resolver/0.log" Jul 08 18:05:36.745468 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:36.745434 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-gw6q5_866f67bd-00c2-4ce4-93a9-36dc09df9314/node-ca/0.log" Jul 08 18:05:37.997649 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:37.997616 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d76f9f859-x65xf_0f8606c0-3e95-4ca5-89db-4af92096ec30/kube-auth-proxy/0.log" Jul 08 18:05:38.166977 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:38.166940 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-829w7_a86ec620-a7c7-49b4-9bb4-29ede056d532/payload-pre-processing/0.log" Jul 08 18:05:38.249701 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:38.249597 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2s86k_d49c818f-9496-433a-9810-7d3bb52d939d/payload-processing/0.log" Jul 08 18:05:39.133630 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:39.133600 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-z2hmv_26cb2c9d-dc86-4938-916d-8151673d9e04/serve-healthcheck-canary/0.log" Jul 08 18:05:39.949865 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:39.949839 2551 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" Jul 08 18:05:39.991565 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:39.991515 2551 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-7l84t/perf-node-gather-daemonset-wl6vt" podStartSLOduration=7.991498357 podStartE2EDuration="7.991498357s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-08 18:05:32 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-08 18:05:33.969027353 +0000 UTC m=+2766.900839091" watchObservedRunningTime="2026-07-08 18:05:39.991498357 +0000 UTC m=+2772.923310075" Jul 08 18:05:40.106322 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:40.106290 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fk242_5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85/kube-rbac-proxy/0.log" Jul 08 18:05:40.152245 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:40.152211 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fk242_5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85/exporter/0.log" Jul 08 18:05:40.186423 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:40.186389 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fk242_5bc76c93-40ad-42c0-a5b2-06c9ab5f5e85/extractor/0.log" Jul 08 18:05:42.407764 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:42.407544 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d76f9f859-x65xf_0f8606c0-3e95-4ca5-89db-4af92096ec30/kube-auth-proxy/0.log" Jul 08 18:05:42.548295 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:42.548264 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-829w7_a86ec620-a7c7-49b4-9bb4-29ede056d532/payload-pre-processing/0.log" Jul 08 18:05:42.616789 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:42.616755 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2s86k_d49c818f-9496-433a-9810-7d3bb52d939d/payload-processing/0.log" Jul 08 18:05:42.740335 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:42.740303 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-5f5468d586-8nlgq_c054a734-9172-404e-a6f7-bc5ee01798fd/manager/0.log" Jul 08 18:05:43.122835 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:43.122766 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-574956cb5f-9czqp_bc48fce2-cb08-4e49-9b71-d8288319f25a/manager/0.log" Jul 08 18:05:43.153608 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:43.153575 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-574956cb5f-9czqp_bc48fce2-cb08-4e49-9b71-d8288319f25a/copy-manifests/0.log" Jul 08 18:05:44.164465 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:44.164423 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7d76f9f859-x65xf_0f8606c0-3e95-4ca5-89db-4af92096ec30/kube-auth-proxy/0.log" Jul 08 18:05:44.298350 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:44.298317 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-7964457696-829w7_a86ec620-a7c7-49b4-9bb4-29ede056d532/payload-pre-processing/0.log" Jul 08 18:05:44.364171 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:44.364128 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-processing-58f86c7798-2s86k_d49c818f-9496-433a-9810-7d3bb52d939d/payload-processing/0.log" Jul 08 18:05:44.554091 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:44.554066 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7b689d6b77-6mlbp_3712c867-b363-4afc-bacf-84dd0d563d3c/manager/0.log" Jul 08 18:05:44.613511 ip-10-0-141-16 kubenswrapper[2551]: I0708 18:05:44.613467 2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-fd8ccff4c-k5hcf_c6176f80-6ae3-4e8e-ab2f-7f215cf1bd55/openshift-lws-operator/0.log"